Large language models often ignore evidence when fact-checking claims
Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation
Computation and LanguageArtificial IntelligenceInformation Retrieval
Summary
Fact-checking systems check if a statement is true using evidence from documents. The authors found that large language models (LLMs), even though good at reasoning, often rely more on what they already know than on the actual evidence given. To test this, they created a method that gradually removes parts of the evidence to see if the models change their answers. They also developed a new training method called REAL that helps models better use the evidence. Their tests showed that REAL-trained models depend more on evidence when verifying facts than regular models do.
fact-checkinglarge language modelsparametric knowledgeevidence groundingevidence ablationcounterfactual evidencemodel trainingverificationnatural language processing
Authors
Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
Abstract
Automatic fact-checking systems assess the veracity of claims given evidence from relevant documents. Large Language Models (LLMs) have demonstrated strong performance in fact-checking due to their general reasoning capabilities. However, it remains unclear whether they faithfully make use of the evidence provided to reach veracity judgments or rely on parametric knowledge. To investigate this, we introduce Fact-Ablated Evaluation (FAE), a new evaluation framework that iteratively ablates the cited evidence to assess whether LLMs revise their predictions accordingly. Our empirical results show that current off-the-shelf LLMs as fact-checking systems rely more on their parametric knowledge than on the evidence provided. To bridge this gap between prediction accuracy and evidence grounding, we propose REAL (Rigorous Evidence Ablation Learning), a training framework that promotes evidence-dependent verification through counterfactual evidence supervision for the LLM-as-verifier models. Experiments on four fact-checking datasets across different domains demonstrate that models trained with REAL obtain superior evidence-dependent capabilities compared to standard fine-tuned models. Our findings highlight that strong fact-checking performance can still coexist with weak evidence dependency, while REAL encourages veracity predictions to remain more closely tied to the availability of supporting evidence.