Before Reasoning Fails: Pre-Evidence Procedural Failures in Agentic RAG
2026-08-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied how AI agents sometimes fail because they grab information but don’t actually read it before giving an answer. They looked at many examples from different question-answering tasks and found that failures can happen either before or after reading evidence, and these failures mostly don’t overlap. To fix this, the authors tested a simple rule making the agent read retrieved information before answering, which improved accuracy significantly. Their work suggests that checking if an agent properly gathers evidence should be treated separately from how it reasons with that evidence.
agentic retrieval-augmented generationevidence-conditioned reasoningHotpotQAtool-call tracesRead-GateLLM accuracytrajectory-level analysisentity extractionlazy retrieval
Authors
Daeyoung Roh, Donghee Han
Abstract
Agentic retrieval-augmented generation (RAG) systems can fail before evidence-conditioned reasoning is tested: an agent may retrieve candidate snippets but finalize without inspecting them. We study this failure mode as a procedural property of the agent trajectory, decomposing wrong answers into pre-evidence discipline failures and post-gold-read failures using saved tool-call traces, retrieved evidence, read passages, and final answers. Across 12,000 paired trajectories on HotpotQA, 2WikiMultiHopQA, and MuSiQue, the two failure types are largely non-redundant: the both-trigger rate is in [11.2%, 13.1%] across regex and spaCy entity extractors. We then evaluate Read-Gate, a minimal runtime invariant requiring an agent to read after search and before finalization. Forced reading improves LLM-Acc by 14.9-19.9 points on trajectories that would otherwise skip reading and by 3.2-9.4 points on full minimal-reasoning cells. Additional diagnostics show that larger hidden thinking budgets do not necessarily increase evidence inspection. Together, these results indicate that evidence-gathering should be evaluated as a trajectory-level control problem, separately from answer-side reasoning.