When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games
2026-07-06 • Computers and Society
Computers and SocietyComputation and Language
AI summaryⓘ
The authors studied whether large language model (LLM) agents stick to what they say they will do when acting in groups. They set up games where agents first decide privately, then announce their intentions publicly, and finally take action, to see if changes happen after announcing. They found that when agents change their announced plans, it usually matches what they had privately intended all along, but this varies by game and model. Also, agents from different models understand announcements differently, some treating them as real promises, others as just talk, which affects game outcomes. This means mixed systems need careful testing to ensure agents interpret commitments the same way.
large language modelsautonomous agentsgame theorycommitmentsprivate intentpublic announcementdeviationhomogeneous groupsheterogeneous groupspayoff
Authors
Jerick Shi, Terry Jingcheng Zhang, Bernhard Schölkopf, Vincent Conitzer, Zhijing Jin
Abstract
As large language models are deployed as autonomous agents that communicate intentions before acting, a critical safety question is whether agents that publicly commit to actions will honor those commitments. We place LLM agents in repeated $n$-player games with a three-stage protocol that separates private intent, public announcement, and final action, allowing us to identify whether each deviation from a stated announcement was already planned during private deliberation. Evaluating three frontier models across six games in homogeneous and heterogeneous groups over 10 rounds, we report two findings. First, when agents deviate from their announcements, the deviation is predominantly already stated in their private plan (exceeding 90% in the highest-deception conditions), yet this is not a fixed model property: the same model ranges from perfect honesty to near-total deviation across games. Second, different models interpret announcements incompatibly, some as binding commitments and others as cheap talk, producing payoff gaps that emerge in Round~0 and persist across all 10 rounds. Systems that combine models from different providers therefore cannot assume shared announcement semantics and require empirical testing of model interactions before deployment.