Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation
2026-08-20 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors explore how language models that solve problems can decide how much effort to spend on each problem instead of always using the same amount of thinking. They let the model pick from three modes: answering quickly, giving a short reasoning, or a long reasoning, and trained it to choose based on the problem difficulty. Their method reduces the amount of thinking needed by 41% on a math test while keeping similar accuracy, and the approach also works well on other problem types without extra training. This shows models can learn to save effort by adapting how much they think for each question.
reinforcement learninglanguage modelstoken budgetreasoning modespolicy optimizationMATH datasetadaptive computationresponse lengthmodel efficiencytransfer learning
Authors
Gijs Kassenaar, Zhao Yang, Vincent François-Lavet
Abstract
Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: \textsc{NoThink} (answer as quickly as possible), \textsc{Short} (brief reasoning), or \textsc{Long} (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than \textsc{Long}, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 ($0.782$ vs.\ $0.796$) while cutting the mean response length from $4{,}796$ to $2{,}811$ tokens (a $41\%$ reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76\% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.