Language models spot lying reporters better than proving honesty
Easy to Catch a Liar, Hard to Clear an Honest One: Language Models Diagnosing a Corrupted Reward Channel from a Verified Record
Machine LearningArtificial IntelligenceComputation and Language
Summary
This paper looks at how artificial intelligence language models figure out if a report of a reward is truthful or lying. It is easy for the models to detect when a reporter lies if given one verified true fact to compare with. However, the models struggle more when trying to prove a reporter is honest, often wrongly calling honest reporters liars. The authors found that these errors depend on seemingly irrelevant details in the test setup. Their research highlights limits in how language models interpret truth from reward reports.
What this means in practice
- •For ai developers: Improve AI system design by understanding limits in identifying honest versus lying reports given a verified data point.
- •For fraud detection teams: Incorporate verified checks with automated analysis to better detect deceit in reported data streams.
Authors
Arman Nik Khah
Abstract
An agent that learns from rewards has to trust whatever reports those rewards. When the reports suddenly change, either the world changed or the reporter broke. From the reports alone these are indistinguishable, and reinforcement learning theory shows that no amount of further experience separates them. The prescribed escape is richer data about the reporter itself. We ask whether a frozen language model, handed exactly that data, uses it. We build a two-option game in which a payout swap and a lying reporter produce byte-identical histories. Then we add one verified record: an independent check of one round's real result, printed beside what the reporter said about that round. That single line settles the case. We ask three large models, from two families, to answer one question with one letter. Is the reporter honest or lying? They catch a lying reporter almost perfectly. At the 70B class that holds in every condition we tried; the 32B model slips in one wording. They clear an honest reporter far less often, and how often depends on things that should not matter. Averaged over rounds, letters, and wordings, a 72B model calls an honest reporter a liar 38% of the time when nothing has changed at all, and 58% of the time when the payouts moved. A 70B model from a second family calls an honest reporter a liar 26% and 48% of the time. The failure is not one of reading, because in the situation where nothing changed the same models score 0.96 to 1.00 with the answer printed in the prompt. Which surface feature drives it differs by family. For the Qwen models it is which round the record names, and for Llama it is which letter stands for "honest." Adding the record to a prompt that already states the answer makes Llama less likely to give that answer. We had registered a prediction for that 58% before the run: 35%. The failure is larger than we expected.