Hybrid reasoning models rely heavily on hidden thinking behavior
Thinking Leakage: A Causal Audit of NoThink Post-Training in Hybrid Reasoning Models
Machine Learning
Summary
When AI models are tuned to think faster using a shortcut mode called NoThink, they seem to get smarter at math problems. But this study shows that much of that improvement comes from the simpler thinking style the model already knew, called Think, slipping through behind the scenes. The researchers measured how much of the gains in NoThink come from this 'thinking leakage' and found it was a big part, sometimes more than half. This means improvements might not be about making NoThink better, but about coaxing out old Think behaviors more effectively.
What this means in practice
- •For ai model developers: Evaluate if performance gains from post-training in NoThink mode are due to new capabilities or hidden Think mode behaviors to better understand model improvements.
- •For math problem solving software teams: Improve inference speed in math reasoning software by identifying and balancing thinking behaviors to achieve faster outputs without losing accuracy.
Authors
Zehao Liu, Vasant G. Honavar
Abstract
Post-training hybrid reasoning models in NoThink mode has attracted growing interest as a way to improve performance while keeping inference fast. However, these gains may draw on thinking behavior already accessible through the base model's Think mode. We formulate this thinking leakage in a causal mediation framework and audit its contribution using bidirectional interventions along a simple base-derived activation direction. Across three models and three post-training methods on competition math benchmarks, we find that leakage is real, causal, and substantial: behavioral and representational analyses reveal shifts toward Think, steering the base model along this direction reproduces most of the post-training accuracy gain, and counter-steering a checkpoint removes a substantial share of what it gains. Across nine aligned checkpoints with positive NoThink gains, the resulting leakage ratio ranges from 42% to 79%. These interventions support a substantial causal contribution of thinking leakage. Our findings show that a post-training method's apparent advantage can therefore reflect greater drift toward Think, obscuring whether it improves capability within NoThink or more effectively re-invokes existing Think behavior.