Large reasoning models use logit masking to reduce overthinking and save time
CALM: Consistency-Aware Logit Masking in Decoding Space for Mitigating Overthinking of Large Reasoning Models
Computation and Language
Summary
Large reasoning models can sometimes overthink problems by taking unnecessary extra steps, which wastes time and can cause mistakes. The authors propose a method called CALM that watches for certain pauses in the model's reasoning and blocks redundant signals to prevent these extra steps. This approach reduces the amount of reasoning needed without hurting accuracy, and sometimes even makes answers more accurate and concise.
What this means in practice
- •For ai system engineers: Improve response speed and reduce compute costs by cutting redundant reasoning steps in advanced reasoning language models.
- •For natural language processing developers: Create more concise and coherent output for math and science reasoning tasks by integrating consistency-aware logit masking into decoding processes.
Authors
Shengjing Liu, Feiyang Li, Yuxuan Yang, Yile Wang, Hui Huang
Abstract
Large reasoning models (LRMs) have achieved strong performance on challenging mathematical and scientific tasks through long chain-of-thought reasoning, but they often suffer from overthinking with unnecessary reasoning steps. Existing truncation-based methods typically detect consecutive trial-answer agreement and force early exit, yet such local stopping signals do not necessarily indicate final-answer stability and can disrupt useful subsequent reflection or verification, potentially leading to performance degradation. We propose consistency-aware logit masking (CALM), a training-free decoding intervention for mitigating overthinking in LRMs. CALM first monitors trial answers at step boundaries and then applies logit masking to action transition point (ATP) tokens such as "Wait", "Alternatively", and "Hmm", suppressing redundant transitional reasoning while allowing the model to continue its thought process more naturally. Experiments across four advanced LRMs and five benchmarks show that CALM reduces token consumption by 18.6%-25.0% compared with full CoT reasoning while maintaining performance, and can even improve accuracy by 0.07%-1.48% by reducing redundant reasoning. LLM-as-a-Judge results also demonstrate that CALM provides more coherent and concise results.