Reliable data is
becoming the
defining advantage
in AI investing
What do investors need to build confidence in AI-enabled workflows while maintaining rigorous standards for data quality,
verification and interpretation?
Data quality still matters
Disclaimer: The Reuters news staff had no role in the production of this content. It was created by Reuters Plus, the brand marketing studio of Reuters. To work with Reuters Plus, contact us here.
Discover the data, research and insights helping investors navigate an evolving investment landscape.
The Reuters news staff had no role in the production of this content. It was created by Reuters Plus, the brand marketing studio of Reuters.
Produced by Reuters Plus for
Analysts and advisors have always needed data that was accurate and timely. The difference now is that generative AI makes it possible to gather data at scale, in a way that simply wasn’t accessible to most investors previously. “Now it seems like almost anyone can be a data analyst,” says Steven. “It is very easy to create the appearance of a high-quality data set.”
This is exactly why Seth sees the importance of having “subject matter experts, like the ones at Morningstar, who can be drawn upon to augment the base competencies of the model with real-time, accurate, curated data, and to gut check the result to make sure that you don't have the hallucinations you can sometimes get from AI model output that is not supplemented with domain-specific knowledge sources.”
Any generative AI model has to be fed with information and data. One way to do that is to survey the web at large. Good AI models provide traceable sources, but then it is up to the user to verify those sources and that exercise can be almost as time consuming as producing the analysis independently, especially if some of those sources are unfamiliar.
The alternative is to use one or more official data connections to providers. “We have worked to produce a dedicated connector to Morningstar data and research that is accessible across leading AI platforms,” says Steven. That way, a user knows exactly where the AI model is sourcing its information, and, for users of Morningstar’s software products or feeds, that source is already familiar and trusted.
It really comes down to traceability and rigor around inclusion and exclusion of sources. Beware the new kid on the block: underlying sources should be credible and backed by decades of experience. Steven warned that this will only become more important as the volume of ‘slop’ generated by and recirculated through AI models continues to grow. “Have you heard of this source before? If not, that’s a big red flag.”
But knowing your source is only one part of verification. “The best implementations of AI are very, very rigorous in defining not only where data may be sourced but also in making sure that attributions to those sources are very clear in deliverable products,” says Seth.
“And it's not enough to know where you got the data, you should know on what date you got the data to prevent uncertainties related to versioning or time-sensitivity. You should know what the query was that produced the result. That provenance tracking is what leads to good implementations.”
Know your sources
Another ‘tell’ that an AI-generated analysis may not have had the proper checks is when there are simply too many words saying too little. AI models will sometimes generate paragraphs to answer simple questions. If you find yourself wading through reams of text to find a response to a simple question, Steven cautioned, that is another red flag.
Beware, too, the vague and non-specific, Seth warned. If the output is general enough, it makes it easy to unwittingly inject your own bias into a result that makes sense to you but might not actually be supported by a more precise analysis.
“Just because investors have more information doesn’t mean they are going to have better outcomes. But there is certainly a great opportunity to provide a lot more transparency into less visible instruments,” says Steven. “Morningstar started out in the business of providing portfolio holdings for mutual funds at a time when they weren't commonly available. So as a firm we are now very focused on how we are using AI to expand the quantity of data we support while making sure that it is still providing a reliable signal for investors.”
Quantity does not equal
quality…
Seth sees the real power of AI not in making the analysis significantly better per se, but in being able to repeat the analysis with more frequency, or in being able to add more variables to the analysis in a way that leads to more durable conclusions than traditional analysis could have.
Think about, say, a pension fund that has made a commitment to a private equity fund and wants to know when the capital is going to be returned so that it can be redeployed. “That's the kind of work that historically was done in spreadsheets and required a ton of different assumptions,” says Seth. “And it would just take a lot of time, maybe weeks, or even months to gather that data and model possible outcomes. Now we can perform that work in a day if not a couple of hours with AI.”
There’s always been a high level of rigor around research content. But now analysts can rerun the model with updated inputs every week, maybe, instead of just once a quarter. “And that inherently leads to better decisions for investors.”
…but speed
improves outcomes
“An AI system is not a thinking entity. It's just a model. It appears as though it's thinking because of the way that it generates responses in a human-like fashion but it’s really just a very sophisticated pattern recognition machine,” says Steven. “An experienced human analyst has the ability to pick apart whether or not the conclusions it's drawing actually make any sense.”
Today’s AI analysis does have limits, and “good practitioners of the technology understand its limits,” Seth acknowledged. For one, Steven notes, “AI is not predictive. AI will always be catching up as information is added to its knowledge set.” Sometimes repetition within that knowledge set can masquerade as certainty, effectively “crystallizing groupthink.” And, because AI models are built to encourage engagement, they will rarely flat out reject user input, which can introduce confirmation bias.
At the end of the day, Steven concluded, with or without AI “firms will still be judged by the same criteria they’ve always been judged by. Are they helping investors meet their financial goals? Is the advisor providing value to clients beyond what they can do on their own? Are they helping clients navigate market questions and life changes? Are institutions responding to actual market demand? Are their products competitive?”
The human
touch
Seth Sprinkle, Global Head of AI Platform Strategy and Partnerships, Morningstar
Artificial intelligence promises faster analysis and greater efficiency for investment professionals. Yet the rapid spread of AI-generated content makes it increasingly difficult to distinguish reliable intelligence from distortion caused by inaccurate or incomplete information. Reuters Plus interviewed two executives at Morningstar who are working to credibly integrate the company’s deep experience in financial data with new AI tools.
Like any article or report, an AI tool’s output is only as reliable as its sources. “Investors expect accuracy,” says Steven Berger, Director of Product Management for Morningstar’s AI Accelerator. Even before AI, “if you were reading an analysis based on fundamentally unverifiable data, that puts the whole thing into question.”
“In some ways, these are the exact same problems that investors have always had,” says Seth Sprinkle, Global Head of AI Platform Strategy and Partnerships at Morningstar.
“Anytime you make a decision based on bad data, or incomplete data, it's going to have negative consequences."
The biggest change that we see with the rise of AI is that a lot of decision making and inference is now behind a curtain, obfuscated behind the model. And so it becomes a lot more difficult to detect some of those pitfalls.
“The data that goes into the AI models is what really differentiates bad, from good, from excellent,” says Seth. “As people become more sophisticated users of AI, they'll quickly recognize that data has become more valuable than ever. And as folks begin to implement a strategy to deploy AI into their organizations part of that conversation has to be what data are we going to use? Are our data providers offering comprehensive support? Are they offering AI-native delivery mechanisms? That will be the difference between success and failure.”
Seth Sprinkle, Global Head of AI Platform Strategy and Partnerships, Morningstar
“Is the data accurate?
And if it’s not, is there
a process to resolve that?”
Steven Berger, Director of Product Management, Morningstar’s AI Accelerator
Seth Sprinkle
“The data that goes into the AI models is what really differentiates bad, from good, from excellent,”
Explore more from Morningstar
Read more
Steven Berger
Director of Product Management, Morningstar’s AI Accelerator