Llm agents benchmarked on forecasting quarterly earnings per share
EPSBench: Benchmarking LLM Agents for Earnings Forecasting
Computation and LanguageArtificial Intelligence
Summary
Forecasting a company's quarterly earnings involves understanding many financial details and how they add up to reported profits. The authors built a test called EPSBench where AI models try to predict quarterly earnings by reading official documents and news about companies. They compare how well these AI models do against professional human analysts. The results show some models perform about as well as analysts, but prediction intervals often underestimate actual earnings. Most errors come from estimating a key income figure called operating income.
What this means in practice
- •For financial analysts: Use EPSBench to test and compare AI tools forecasting quarterly earnings against analyst benchmarks.
- •For investment software developers: Incorporate EPSBench to evaluate and refine LLM-based earnings prediction features in financial products.$Commercial implications: EPSBench enables creating improved AI-driven earnings forecasting tools for investment platforms, offering competitive advantage.
Authors
Yanbin Yin, Chuyu Xu, Liqin Ye, Jing Peng, Chao Zhang
Abstract
Quarterly earnings forecasting requires an agent to connect financial evidence to the income-statement line items that produce a company's reported earnings. We introduce EPSBench, a benchmark for studying this ability across 149 firm-quarters from 86 US companies. Agents read a fixed collection of filings, earnings-call transcripts, and news, then estimate company-specific income-statement drivers with cited evidence. A shared accounting calculator converts these estimates into GAAP diluted earnings per share (EPS), providing a common basis for comparing forecast assumptions. We evaluate six models under a common harness, assessing point forecast error relative to date-matched analyst consensus and checking whether their prediction intervals achieve the stated coverage. Three models show no detectable paired difference in point forecast error from consensus, and three have higher error. Across all six models, nominal 80% intervals contain reported EPS in only 37--70% of runs that return an interval, and most misses fall above the upper bound because the forecasts run low. The driver records trace most of the error gap between models to operating-income estimates.