A finance professor at the University of Florida, Alejandro Lopez-Lira, has used the large language model, ChatGPT, to parse news headlines and forecast stock prices. According to his recent paper, ChatGPT’s ability to predict the direction of the next day’s returns was better than random. However, the experiment also shows the limitations of large language models in finance tasks. Lopez-Lira predicts that ChatGPT’s ability to predict stock moves may decrease in the coming months as institutions start integrating this technology.
Alejandro Lopez-Lira, a finance professor at the University of Florida, has suggested that large language models like ChatGPT could be used to forecast stock prices. Lopez-Lira used ChatGPT to parse news headlines for whether they’re good or bad for a stock, and discovered that ChatGPT’s ability to predict the direction of the next day’s returns was much better than random.
According to Lopez-Lira, the ability of ChatGPT to understand headlines from financial news and how they might impact stock prices could put high-paying jobs in the financial industry at risk. In a report published on March 26 by Goldman Sachs, the company predicted that about 35% of banking employment was vulnerable to automation.
However, the experiment also shows how far large language models are from being able to do many finance tasks. The experiment did not include target prices or have the model do any math at all. In fact, ChatGPT-style technology often makes up numbers, as Microsoft learned in a public demo earlier this year.
Lopez-Lira said he was surprised by the results, adding that they suggest that sophisticated investors aren’t using ChatGPT-style machine learning in their trading strategies yet. He also noted that the model’s ability to predict stock moves may decrease in the coming months as institutions start integrating this technology.
In the experiment, Lopez-Lira and his partner Yuehua Tang looked at over 50,000 headlines from a data vendor about public stocks on the New York Stock Exchange, Nasdaq, and a small-cap exchange. They fed the headlines into ChatGPT 3.5 along with the following prompt: Then they looked at the stocks’ return during the following trading day.
Furthermore, Lopez-Lira found that when the model was given a news headline, it performed better virtually every time. He discovered that the likelihood that the model would do equally well when choosing the next day’s move at random versus when it was guided by a news item was less than 1%.
ChatGPT also beat commercial datasets with human sentiment scores. One example in the paper showed a headline about a company settling litigation and paying a fine, which had a negative sentiment, but the ChatGPT response correctly reasoned that it was actually good news, according to the researchers.
Lopez-Lira told CNBC that hedge funds had reached out to him to learn more about his research. He also said it wouldn’t surprise him if ChatGPT’s ability to predict stock moves decreased in the coming months as institutions started integrating this technology.
“As more and more people use these types of tools, the markets are going to become more efficient, so you would expect return predictability to decline,” Lopez-Lira said. “So my guess is, if I run this exercise, in the next five years, by the year five, there will be zero return predictability.”
