Fact-checking agents handle source trust more reliably with new training method

Multi-Channel Mitigation of Source-Trust Shortcuts in Fact-Checking RL Agents

Artificial Intelligence

Summary

Fact-checking models often use trust labels like HIGH or LOW to decide how confident they are in information sources. The authors found that sometimes these labels can trick the model into changing its verdict unfairly. They created a test called TrustSwap to check how models react when source labels are changed without altering the evidence. Then, they developed a training method called trust-swap augmentation to reduce this problem, helping models rely on evidence content instead of just source labels. This approach works well for smaller models but needs more study at larger scales.

What this means in practice

  • For fact-checking system developers: Improve fact-checking agents to avoid overreliance on source trust labels and make more evidence-based verdicts by applying trust-swap augmentation during training.
  • For ai model trainers: Use trust-swap data augmentation to reduce source-trust shortcuts in reinforcement learning fine-tuning of language models aimed at fact verification tasks.

Authors

Jianchang Su, Yiwei Yang, Wei Zhang

Abstract

Retrieval-augmented fact-checkers often receive a reliability label, such as HIGH or LOW trust, for each evidence source. These labels should adjust the model's confidence and its decision to search for more evidence, while the verdict should follow the evidence content. We introduce TrustSwap, a counterfactual test that swaps, lowers, or removes source labels while keeping every evidence text fixed, and measures its three output channels (the verdict, the confidence, and the search decision) separately. Across untrained and RL-trained models at two scales, three datasets, and two prompts, confidence and search respond to the labels as intended in 49 of 50 comparisons, yet a label change alone alters 4-23% of confident verdicts for Qwen3 models and up to 50% for an existing RL-trained fact-checker. Standard GRPO fine-tuning amplifies this shortcut at 8B in all six settings. To reduce it, we propose trust-swap augmentation (TSA), which trains GRPO on each claim with both its original and its label-swapped evidence under the same gold verdict. At 4B, TSA lowers the verdict flip rate by 7-35% (relative) in four of six settings, keeps accuracy and the intended confidence and search responses, outperforms reward-based alternatives in the main setting, and carries over to an unseen label-removal perturbation. An added consistency reward helps on the trained-on swap but not on unseen perturbations. At 8B, TSA's effect is not detectable, which makes scale the main open question.