Pruned large reasoning models fail to stop but can be fixed
To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models
Computation and Language
Summary
Large reasoning models are often made smaller by pruning their weights, but this can cause them to get stuck repeating themselves instead of stopping after finding the correct answer. The authors discovered that these repeated loops happen even when the model knows it should stop, so the problem isn’t reasoning but stopping. They identified a specific pattern in the model’s internal signals linked to deciding to stop, which stays consistent across pruned versions. Using this insight, the authors created a simple method to guide the model to stop at the right time during use, improving its accuracy and reducing repetitive outputs without retraining.
What this means in practice
- •For ai model deployers: Improve performance of pruned large reasoning models by reducing repetitive loops and increasing correct stopping behavior during inference.
- •For software engineers for chatbot systems: Enhance chatbot reliability by applying a method that guides pruned reasoning models to end responses promptly after finding correct answers.
Authors
Minh-Anh Nguyen, Nguyen T. Ha Lan, Xiyan Fu, Dung D. Le, Wei Lu
Abstract
Weight pruning can degrade the accuracy of large reasoning models (LRMs), yet it remains unclear whether it damages reasoning itself. We find that much of this degradation stems from a failure to stop rather than to reason. As sparsity increases, pruned LRMs increasingly fail to terminate within the token budget, often after the correct answer has appeared in their reasoning, instead falling into repetitive loops, even though they still recognize where the dense model would stop. We further find that the hidden-state difference between terminating and looping continuations is consistent across pruned checkpoints, suggesting a shared direction associated with committing to an answer. Building on this, we propose Termination-Guided Activation Steering (TGAS), a training-free method that adds this direction to the residual stream at inference time. Across six LRMs, three pruning pipelines, and math and code benchmarks, TGAS improves termination and overall accuracy while reducing repetitive reasoning and generation length, showing that this failure is actionable.