Misinformation fact-checking scores inflated by future knowledge leaks
No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection
Computation and Language
Summary
Fact-checking algorithms that judge the truth of online claims sometimes look better than they really are. This happens because they can use information that wasn’t actually available when the claim was first made, like facts learned later or evidence published after the claim. The authors studied this problem and found that when tests don't control for this, fact-checking models can appear more accurate than they truly should be. They also discovered a simple way to reduce this 'future knowledge' problem in the models.
What this means in practice
- •For social media platform engineers: Improve evaluation methods for automated fact-checkers by accounting for information leakage from future evidence and pretrained knowledge.
- •For fact-checking tool developers: Design fact-checking systems that reduce reliance on post-claim evidence to provide more realistic real-time misinformation detection.
Authors
Kuan-Hua Wu Lu, Yohanes Andre Setiawan
Abstract
As automated fact-checking scales on social media, large language model (LLM) verdict scores can look stronger than warranted. One reason is that evaluations mix in information that was not knowable at claim time. Two channels are easy to conflate: outcomes memorized in pre-training and retrieved evidence published after the claim. Yet standard benchmarks rarely separate the two. In this study we measure both channels on AVeriTeC and QuanTemp++ by reconstructing point-in-time evidence conditions and probing for outcome information encoded in model representations. We find substantial evidence of parametric leakage, that can be hidden by the aggregate accuracy, while a simple representation bottleneck reduces this future leakage more efficiently than a mutual-information-based training penalty. We also find that allowing post-claim evidence inflates zero-shot accuracy by 6.3 points in AVeriTeC while the effect is negligible in QuanTemp++, where retrieval provides little post-claim evidence. These results show that misinformation benchmarks can overstate fact-checking performance when they do not account for what information was actually available at claim time.