Predicting when more compute improves mathematical reasoning accuracy
When Does Longer Reasoning Help? Predicting Mathematical Reasoning Through Discovery and Execution
Machine LearningComputation and Language
Summary
Mathematical problem-solving by AI can often improve if given more computation time, but it’s hard to know in advance how much extra effort will help. The authors created a new approach that looks at how an AI finds strategies and then executes them to predict how well it will do with more or less computing. Testing on challenging math problems showed that simply guessing based on early success isn’t always accurate, but their method gives better predictions of future performance. This means it could help decide whether to give AI more time or try a different approach during problem solving.
What this means in practice
- •For ai system builders: Optimize allocation of compute resources to improve AI problem-solving efficiency using predicted scaling of reasoning success.
- •For automated theorem proving teams: Improve decision-making on continuing or restarting reasoning processes to reduce wasted computation when solving complex math tasks.
Authors
Adib Hasan, Lay Jain, Thanic Nur Samin
Abstract
Test-time compute can improve mathematical reasoning, but can short-budget runs predict how mathematical reasoning scales with additional compute? We introduce a Discovery--Execution (DE) framework that predicts the aggregate held-out scaling curves through a convolution of strategy discovery and conditional execution. From independent short-budget attempts and oracle-sketch-conditioned runs, the framework estimates cumulative success along held-out reasoning trajectories under alternate compute allocations. We evaluate four models on 35 fresh Olympiad problems and non-geometry problems from IMO-ProofBench Advanced. Under the DE framework, near-saturated execution predicts geometric scaling, as observed for the GPT models. For Claude Opus 4.8, incorporating measured execution substantially improves held-out forecasts over geometric extrapolation across one- and two-arm allocations. As a secondary application, regularized DE (R-DE) decisions to continue or restart yield lower average regret than the best model-specific retrospective policy. Together, these results show that measuring conditional execution provides information about longer reasoning that short-budget success rates do not always capture.