Large language models improved for numerical claim checks
Towards Robust Numerical Claim Verification
Computation and Language
Summary
Verifying claims that include numbers is tricky for large language models (LLMs) because even small number changes can cause mistakes. The authors found that this problem still happens in the newest LLMs but can be fixed by fine-tuning them with examples where numbers are deliberately changed. This training method helps smaller models outperform bigger, popular ones and even works well in different languages without extra data. Their approach also improves handling cases where the evidence itself has tricky changes.
What this means in practice
- •For fact-checking teams: Enhance reliability of automated tools verifying numerical claims by reducing errors from small value changes.
- •For multilingual ai developers: Improve numerical claim verification accuracy across languages without requiring additional target-language data.
Authors
Peter Røysland Aarnes, Vinay Setty
Abstract
Large language models (LLMs) are widely used for claim verification, yet remain brittle for numerical reasoning: even small changes in value can sharply degrade accuracy. We show that this brittleness persists in frontier LLMs, but can be mitigated through adversarial fine-tuning on numerically perturbed examples. Using parameter-efficient fine-tuning, small Qwen3 models (0.6B$\unicode{x2013}$8B) reach 98.7% accuracy on label-flipping perturbations, outperforming larger zero-shot models and frontier systems (GPT-5.4 Pro (74.0%) and Gemini 2.5 Flash (73.9%)). The gains generalise to unseen perturbation types, indicating robust numerical decision boundaries rather than memorised edits. Robustness also transfers without target-domain data, significantly improving cross-lingual performance in Spanish. We further show that the same fine-tuning recipe confers robustness to evidence-side perturbations, using the VitaminC dataset.