Language model forecasts use momentum and tone more than earnings surprises
Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts
Machine LearningArtificial IntelligenceComputation and Language
Summary
The study looks at what influences large language models when they predict company earnings. The researchers gave the models made-up earnings reports that changed key factors like recent stock momentum, the tone of commentary, and actual earnings surprises. They found that models rely more on recent momentum and the tone of commentary than on the actual earnings surprises. The work shows how language models use these inputs but does not claim they internally reason or can predict stock returns.
What this means in practice
- •For financial technology teams: Improve language model tuning by focusing on momentum and commentary tone to enhance earnings forecast inputs.
- •For natural language processing engineers: Design testing setups that isolate input factors like tone and momentum to better understand model behavior on financial text.
Authors
Alina Khaybullina
Abstract
Which inputs drive a language model's financial forecast? We construct synthetic earnings announcements that independently reverse 20-day momentum, non-informational commentary tone, and whether fixed actual EPS beats a manipulated consensus. Six instruction/chat checkpoint conversions from four families (0.49-7.61B base parameters) receive 2,016 prompts each under a direct-answer Q8_0 protocol. The design includes 24 numerical blocks, six commentary pairs, profits and losses, rising and falling annual EPS, and two surprise sizes. Both A/B answer assignments and U/D cover all 384 primary scenarios per conversion. We estimate matched changes in conditional answer-prefix scores, not calibrated market probabilities. Across these checkpoints, the primary EPS contrast ranges from -0.19 to 86.80 percentage points. The magnitude of the primary signed tone contrast exceeds that of EPS in 5 of six models, and momentum exceeds it in 5. Natural-word numerical diagnostic accuracy ranges from 51.0 to 95.8 percent. Answer-encoding, wording, context, and conversion controls qualify these results. The experiment measures observable input reliance and numerical recognition; it establishes neither an internal reasoning mechanism nor realized return predictability. Public benchmark prompts, recorded scores, and an interactive recorded-results explorer accompany the study.