HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning
2026-08-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors introduce a new method to help large language models (LLMs) figure out where their reasoning went wrong. Instead of looking at reasoning steps one by one or as a simple list, their approach uses a hypergraph to capture the complex connections between steps. By comparing a failed reasoning path to a successful one, their method quickly finds the root cause of errors. They train a smaller model with this data, which then helps improve reasoning accuracy on tasks like math and coding.
large language modelsreasoningfailure attributionhypergraphcounterfactual testingsupervised fine-tuningreinforcement learningmathematical reasoningagentic coding
Authors
Runchuan Zhu, Hongbin Lai, Bowen Jiang, Junrui Zhang, Zhangheng LI, Ostap Kilbasovych, Junyuan Hong
Abstract
Reflection is a powerful mechanism for LLM reasoning, yet its effectiveness hinges on accurately attributing failures to specific reasoning steps, a capability that current models notably lack. Existing failure attribution methods either require expensive step-by-step counterfactual testing that scales poorly with trajectory length, or treat reasoning traces as flat sequences that ignore the inherent non-linear logical dependencies. We propose a hypergraph-based paired failure attribution (HPFA) framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path. By reducing the search space, our method efficiently localizes root causes and enables scalable synthesis of attribution data for training a lightweight attributor model via supervised fine-tuning and reinforcement learning. Experiments on mathematical reasoning and agentic coding tasks demonstrate that HPFA can dramatically increase attribution accuracy and efficiency, and the trained attributor consistently improves reasoning accuracy at test time, outperforming baselines that lack graph structure or paired analysis.