Beliefrag improves multi-step question answering by tracking evidence state
BELIEFRAG: Making Adaptive RAG State-Aware under Evolving Evidence
Artificial Intelligence
Summary
When computers answer complex questions by searching for information in several steps, they can lose track of what they already know, what’s missing, and when to stop searching. The authors created BELIEFRAG, a system that keeps a clear record of the evidence quality and gaps during the search process, so it can make smarter choices about when to look for more facts, rephrase questions, or give an answer. This approach made the computer answer questions better while using fewer words overall. The biggest improvement came from fixing mistakes by searching again, and making the system’s confidence levels more reliable helped it decide better.
What this means in practice
- •For conversational ai developers: Improve multi-step question answering systems by maintaining a clear evidence state to select when and how to retrieve supporting information.
- •For enterprise search engineers: Enhance search systems that gather iterative evidence by using a belief state to reduce query costs while maintaining answer quality.
Authors
Hongji Pu
Abstract
Adaptive RAG uses signals such as confidence, relevance, support, and retrieval quality to decide when to search or correct evidence. In multi-step retrieval, however, these local signals must be combined into a persistent view of what the current evidence supports, what remains missing, and which action should follow. Existing methods often use such signals as separate triggers, making it difficult to preserve a coherent evidence state across a trajectory; we call this problem evidence-state fragmentation. We introduce BELIEFRAG, a closed-loop controller that updates an explicit state over sufficiency, reliability, conflict, uncertainty, evidence gaps, and acquisition cost, then chooses among retrieval, query rewriting, verification, answering, stopping, and abstention. Across six QA benchmarks with gpt-oss-120b, BELIEFRAG reaches mean token F1 0.572 with 3.89k tokens per question, outperforming fixed iterative retrieval (0.555 F1) while using 39% fewer tokens. The same quality-cost pattern transfers to Qwen3-32B, where BELIEFRAG reaches 0.552 F1 versus 0.523 for iterative retrieval while using 35% fewer tokens. Analysis shows that the main gains come from corrective re-retrieval rather than pruning alone, while several belief dimensions are redundant and calibrated answerability plays the strongest operational role. Calibration improves threshold stability across related evidence sources, although source shift can still invalidate the same decision signal.