Belief based method cuts reasoning steps in large language models
Same Beliefs, Different Policies: Learning Reusable Representations for Early Stopping in LLM Reasoning
Artificial IntelligenceMachine Learning
Summary
Large language models (LLMs) that solve problems often keep thinking even after finding the right answer, which wastes time and computing power. The authors present a new method called BELT that helps the model decide when to stop reasoning by predicting if continuing will improve the answer. BELT learns from the model’s internal signals and can adapt to different trade-offs between accuracy and cost without retraining. This approach reduces the number of reasoning steps by about half without losing accuracy, and it works across different problem types like math, coding, and agent tasks.
What this means in practice
- •For llm engineers: Implement BELT to reduce computation by stopping reasoning early without retraining for each cost preference.
- •For software developers: Deploy LLMs that use BELT to speed up code generation and interactive agent responses by cutting unnecessary reasoning.
Authors
Jingwei Ni, Dmitry Abulkhanov, Vladislav Smirnov, Markus Leippold, Elliott Ash, Timothy Baldwin, Artem Shelmanov, Mrinmaya Sachan
Abstract
Reasoning LLMs often continue reasoning after reaching a correct answer, wasting computation and sometimes replacing it with an incorrect one. Deciding when to stop is a metareasoning problem: the model must weigh what further reasoning could achieve against its cost, given what it can already answer. Direct utility training learns a stopping representation at a chosen token price; supporting new prices can require additional controller training. We introduce BELT (BELief-based Termination), which separates outcome prediction from stopping control. It learns beliefs about current and future answers from a frozen LLM's hidden states, then fits a linear stopping policy on the frozen features. The prediction targets are independent of token price, so one probe supports different accuracy-cost trade-offs through CPU policy refitting or threshold adjustment. Averaged over five out-of-distribution (OOD) math benchmarks, BELT reduces generated tokens by 41% on Qwen3-1.7B and 51% on OLMo-3-7B-Think without loss of accuracy. The same math-trained BELT transfers to coding and agentic tasks without training on code or agentic data. In all three domains, its accuracy-cost trade-offs are competitive with or better than those of end-to-end controllers. On OLMo3, its probe has 21.2x fewer parameters than one checkpoint of the strongest baseline, OS-Pruner, and needs 55x fewer training GPU-hours than OS-Pruner's five price-specific checkpoints.