Efficient method improves fact checking of claims from documents

Label-Efficient Document-Grounded Factuality Detection via Cross-Paraphrastic Invariance Learning

Computation and LanguageArtificial Intelligence

Summary

Sometimes, computer programs make claims based on documents that are not actually supported by the information in those documents. The authors present a new approach that helps detect whether a claim truly matches the facts in a document but requires very few labeled examples to learn from. Their method creates pairs of claim-document examples by rephrasing claims and comparing ones with opposite labels to teach the model better. It performs well on a standard benchmark while needing much less labeled data and runs efficiently when checking claims.

What this means in practice

  • For fact-checking teams: Improve automated detection of unsupported claims in documents using fewer labeled examples, speeding up the review process.
  • For content moderation teams: More efficiently flag potentially false claims grounded in source documents to support content review decisions.

Authors

Chao Chen, Shanshan Lin, Dongsheng Hong, Sibo Ju, Wenjie Yang, Sihong Xie, Xiangwen Liao

Abstract

Large language models (LLMs) can generate fluent claims that are unsupported by a given source document, motivating accurate and efficient document-grounded factuality detection. Existing LLM-based evaluators often require repeated inference and incur high computational cost, whereas compact factuality detectors are efficient at inference time but commonly rely on large labeled or synthetically generated corpora. We propose CPIL (Cross-Paraphrastic Invariance Learning), a two-stage Siamese framework that converts a small labeled set into relational supervision without acquiring additional factuality labels. CPIL constructs positive pairs from distinct label-preserving paraphrastic views of the same document-claim instance and negative pairs from opposite-label examples associated with the same source document. It further introduces quantile-aware semi-hard negative mining, which derives a boundary from the positive-pair similarity distribution and balances informative negatives with broader coverage. In the two-stage training procedure, Stage 1 learns an encoder from these task-aware pairs, and Stage 2 jointly fine-tunes the encoder and a linear classification head for binary factuality prediction. At inference time, CPIL requires no paraphrase generation, pair construction, or Siamese comparison and uses only one encoder-classifier forward pass. On the 11-task LLM-AggreFact benchmark, CPIL-2% and CPIL-3% achieve macro-F1 scores of 81.04 and 81.60 while using only 2-3% of the FactCG-released labeled pool. Both settings outperform all compared compact detectors and two vanilla LLM references. Ablations and sensitivity analyses support the contributions of task-aware pair construction, dynamic negative mining, and multi-pivot paraphrasing.