New method finds where reasoning errors really begin in AI answers
The Error You See Is Not the Error You Made: Progression-aware Reasoning Origin for Reasoning Error Localization
Artificial Intelligence
Summary
Sometimes when AI systems make multi-step reasoning mistakes, the first wrong step isn't obvious because errors can appear only later on. The authors propose a new way to find exactly where the mistake started, not just where it was noticed. Their method carefully looks at how each step fits with what came before and what follows. Tests show it works better than older ways to spot the true origin of errors in reasoning chains.
What this means in practice
- •For ai product developers: Pinpoint the exact step causing faulty AI reasoning to improve model debugging and trustworthiness.
- •For medical informatics teams: Detect the initial error in AI-assisted medical diagnosis steps to enhance decision accuracy and safety.
Authors
Yiguo Wang, Ziyuan Yang, Yi Zou, Dan Lin, Rongsheng Li, Yi Zhang
Abstract
Verifying multi-step LLM reasoning requires more than determining whether a trace is correct: a useful verifier should identify where the reasoning first goes wrong. However, existing holistic methods provide little positional evidence, while forward sequential verification often treats the first rejected step as the error source. Under error propagation, this assumption can fail, since an earlier mistake may remain locally plausible and become observable only through its downstream consequences. We therefore rethink reasoning verification as a progression-aware error-source localization problem: rather than asking only where a reasoning trace first appears inconsistent, we ask which earlier step best explains how that inconsistency emerges along the trajectory. Based on this view, we propose Progression-aware Reasoning Origin (PRO), a training-free framework for first-error localization. PRO jointly models incoming support from the preceding context and outgoing compatibility with subsequent reasoning, selectively refines regions where these signals disagree, and finally performs detector-conditioned source attribution with intervention-based evidence to distinguish the true error origin from its propagated manifestations. We further formalize the gap between forward rejection and structural exposure, showing why incoming-side evidence alone is insufficient for reliable localization under error propagation. Experiments across open-form, medical, and structured reasoning tasks demonstrate consistent improvements over strong verification baselines, supporting progression-aware source attribution as a more faithful formulation of reasoning verification.