MicroMouse improves long chain reasoning by masking bad steps
MicroMouse: Masking Harmful Supervision in Long CoT via Relative Potential
Computation and Language
Summary
Sometimes, when AI models solve problems step-by-step (called chain-of-thought), parts of their reasoning can be misleading or unhelpful. The authors compare this reasoning to a maze-solving mouse that tries to reach the goal. They create a way, called MicroMouse, to measure how close each reasoning step is to the correct answer by comparing it with a similar wrong answer. This helps identify and hide parts of the reasoning that lead away from the right solution, improving the model’s accuracy and efficiency.
What this means in practice
- •For ai system developers: Improve accuracy and output length of AI systems that generate long reasoning steps by masking harmful supervision automatically.
- •For technical writers: Generate clearer and more reliable multi-step explanations by reducing redundant or erroneous reasoning sequences in AI-generated chains of thought.
Authors
Xiaowei Zhu, Hengzhu Tang, Zhenyu Zhang, Shuohuan Wang, Yubing Ren, Yanan Cao
Abstract
Even successful long CoTs may contain harmful supervision, motivating the need for methods that can automatically identify and mask such signals during SFT. Inspired by the MicroMouse maze-solving process, we view a long CoT as a search trajectory from the input problem toward the target answer. This perspective motivates maintaining a goal-relative measure, analogous to a MicroMouse's spatial coordinates, to quantify how close the current reasoning state is to the target answer. Building on this perspective, we propose MicroMouse, an automated selective-SFT framework for masking harmful supervision in long CoTs. In MicroMouse we introduce a Relative Potential (REP) to measure how closely each reasoning state supports the gold answer. Unlike the perplexity of the gold answer alone, REP measures relative support by contrasting the gold answer with a matched foil. This contrast cancels likelihood variations shared by the pair, yielding a more stable signal of progress toward the target answer. By tracking the REP trajectory, MicroMouse identifies consecutive potential decreases as erroneous exploration and post-peak reasoning as redundant wandering, and masks these harmful supervision during SFT. Extensive experiments across multiple student model scales and reasoning benchmarks show that MicroMouse consistently outperforms Full-CoT SFT and existing selective-SFT baselines, yielding average relative gains of 5.5% in greedy accuracy and 2.6% in Pass@1 relative to Full-CoT SFT. Meanwhile, MicroMouse improves reasoning efficiency by reducing average output length by 8.8%, while identifying harmful supervision in only 1.7 seconds per long CoT on average.