Summary
Fact-checking systems aim to verify claims using evidence, but it is unclear if large language models (LLMs) actually use that evidence or just rely on what they already know. The authors created a test called Fact-Ablated Evaluation that gradually removes pieces of evidence to see if LLMs change their answers. They found that many LLMs tend to ignore the evidence and depend mostly on their stored knowledge. To fix this, they introduced a training method called REAL to teach LLMs to pay closer attention to the evidence when verifying facts. Their experiments show this method helps models rely more on provided evidence while still making accurate judgments.
What this means in practice
- •For fact-checking teams: Improve fact-checking systems by training models to depend more on supplied evidence instead of memory alone during claim verification.
- •For content moderation teams: Develop more reliable AI tools that evaluate online claims based on actual evidence rather than relying on general knowledge.
Authors
Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
Abstract
Automatic fact-checking systems assess the veracity of claims given evidence from relevant documents. Large Language Models (LLMs) have demonstrated strong performance in fact-checking due to their general reasoning capabilities. However, it remains unclear whether they faithfully make use of the evidence provided to reach veracity judgments or rely on parametric knowledge. To investigate this, we introduce Fact-Ablated Evaluation (FAE), a new evaluation framework that iteratively ablates the cited evidence to assess whether LLMs revise their predictions accordingly. Our empirical results show that current off-the-shelf LLMs as fact-checking systems rely more on their parametric knowledge than on the evidence provided. To bridge this gap between prediction accuracy and evidence grounding, we propose REAL (Rigorous Evidence Ablation Learning), a training framework that promotes evidence-dependent verification through counterfactual evidence supervision for the LLM-as-verifier models. Experiments on four fact-checking datasets across different domains demonstrate that models trained with REAL obtain superior evidence-dependent capabilities compared to standard fine-tuned models. Our findings highlight that strong fact-checking performance can still coexist with weak evidence dependency, while REAL encourages veracity predictions to remain more closely tied to the availability of supporting evidence.