Large language models predict stock returns but struggle with earnings forecasts

From SEC Filings to Financial Signals: Evaluating LLMs on Stock Market Reactions and Analyst Forecasts

Artificial Intelligence

Summary

Financial reports called Form 10-Q contain lots of detailed company information, but it’s unclear if AI language models can use that to predict how stocks will perform or what analysts expect. The researchers tested 13 language models on nearly 46,000 reports, checking if their outputs matched stock returns and analyst forecasts. They found that while some finance-focused models were good at ranking stocks on the day of the report, no model consistently forecasted earnings better than human analysts. Adding extra fundamental data improved predictions slightly, but analyst summaries did not help much.

What this means in practice

  • For financial data analysts: Use large language models combined with fundamental data to better rank stocks around financial disclosures for short-term market insights.
  • For investment research teams: Incorporate finance-specialized language models to supplement event-date stock return predictions but rely on analysts for accurate earnings forecasts.

Authors

Shengkun Wang, Taoran Ji, Linhan Wang, Sattar Mansi, Chang-Tien Lu, Yinlin Chen

Abstract

Form 10-Q MD&A sections combine long-form management narrative with accounting and forward-looking information. Benchmarks for financial filings commonly evaluate question answering or numerical reasoning, leaving unclear whether outputs from large language models (LLMs) align with market and analyst outcomes. We evaluate prompted, zero-shot outputs from 13 general-purpose and finance-specialized LLMs on 45,964 Form 10-Q filing events from 2001 to 2024, linking MD&A text to cross-sectional fundamental ranks, pre-filing analyst forecasts, and realized abnormal returns. The evaluation covers return ranking under five controlled information views and next-quarter EPS forecasting under direct and consensus-anchored interfaces. Relative to disclosure alone, adding cross-sectional fundamental ranks raises mean effective-date rank correlation (RankIC) by 0.0089 for effective-event-date returns and 0.0098 for following-day returns, whereas analyst-summary context provides no comparable average gain. Across the four disclosure-containing views, finance-specialized models show an effective-event-date group advantage but no statistically reliable following-day advantage, while general-purpose Qwen models produce the strongest individual return-ranking results. Under the full-information view, direct LLM forecasts average the 43.4th percentile of the analyst distribution and do not consistently beat individual analysts. Consensus anchoring raises the mean inside-range share from 41.1% to 59.5% but increases reported-EPS error.