Active evidence checking improves fact verification accuracy
Stop or Clarify? Experience-Conditioned Active Fact Verification
Artificial Intelligence
Summary
Fact-checking systems often decide whether a claim is true or false based on gathered evidence, but it's unclear when they should stop looking for more information. The authors propose a method called SEAC that helps systems decide if they've found enough evidence or if they should ask for more. This method learns from past experience to balance the benefit of clarifying with the cost of searching further. Tests show SEAC improves accuracy on popular fact-checking tasks by selectively gathering more evidence only when needed.
What this means in practice
- •For fact-checking teams: Improve automated claim verification by adding a selective step to gather more evidence only when beneficial, reducing false conclusions.
- •For legal research teams: Enhance document retrieval by dynamically deciding when to seek additional case law or evidence for claims based on past outcomes.
Authors
Yuhang Dong, Jinghong Li, Xinyue Wang, Yuanning Cui
Abstract
Fact-verification benchmarks score the final label, while an interactive verifier must first decide whether the available evidence is sufficient. We focus on that intermediate Stop-or-Clarify Decision. Uncertainty alone is insufficient, because the benefit of another evidence-acquisition step is unknown before the step is taken. Sequential Experience-Adaptive Clarification (SEAC) wraps a fixed verifier with a lightweight evidence-acquisition policy. For each claim, SEAC retrieves evidence, runs the verifier, forms a label-free decision state, and combines stop risk with an observational clarification-utility estimate from earlier executed trajectories. Clarification is requested only when this score exceeds the action cost and decision threshold. On SciFact, LIAR, and FEVER, SEAC improves mean accuracy and macro-F1 over the compared fact-verification and active-retrieval baselines. Equal-budget and formula-level diagnostics support selective clarification as a contributor to the gain, while the small SciFact split leaves its paired improvement inconclusive.