Correctness signals steer reasoning in language models math tasks
COMPASS: Finding Where Reasoning Lives in Language Models
Artificial Intelligence
Summary
Large language models can solve math problems better if you guide them to think through the steps first, but existing methods need you to know exactly how reasoning looks. The authors show that just checking whether the model's own answers are right or wrong can reveal a hidden 'reasoning direction' inside the model's brain. They developed COMPASS, a method that finds certain parts of the model responsible for reasoning and nudges them to think more correctly, improving math problem-solving without extra training. This approach works well across different models and math datasets, making the reasoning process more efficient and accurate.
What this means in practice
- •For ai engineers: Improve math problem-solving accuracy of language model deployments by steering the model’s reasoning-related activations during inference.
- •For natural language processing teams: Increase efficiency in generating correct multi-step reasoning outputs with fewer tokens by applying correctness-based activation steering.
Authors
Pratyay Dutta, Kowshik Thopalli, Vivek Narayanaswamy
Abstract
Explicitly eliciting reasoning substantially improves LLM performance. Existing approaches require a predefined characterization of reasoning, whether through CoT prompt design, contrastive CoT directions, or via SAE derived reasoning features. For mathematical reasoning with verifiable answers, we show that a much simpler signal suffices, which is the correctness of the model's own direct answer attempts. This signal yields a latent direction that elicits reasoning. This direction is decodable within the activations of most attention heads, but only a small subset of them can be effectively intervened. We introduce COMPASS, an inference-time steering method that identifies these heads using a logit-space attribution score and steers their activations along the correctness direction, requiring only per-head activation statistics. Across three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines we compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70\% fewer generated tokens. Interventions transfer without re-fitting to unseen benchmarks, and ablations show that both the correctness direction and the small set of heads carrying it are necessary, with the effect concentrated in remarkably few heads.