Large language models change game outcomes with unclear numerical signals
When Numbers Start Talking: Numerical Signalling and Strategic Behaviour Among LLMs
Artificial Intelligence
Summary
When multiple AI language models interact in games that require cooperation, the kinds of messages they send affect how well they work together. The authors find that messages with structured information, like numbers or language, change game results but not in a predictable way. Sometimes the AI agents develop number signals linked to game rewards, yet these signals are hard for humans to understand. This means that watching AI behavior alone isn’t enough to track cooperation, and closer attention to the message patterns is needed.
What this means in practice
- •For ai system developers: Monitor and detect coordination among AI agents by analyzing message patterns instead of only their actions in strategic environments.
- •For security operations teams: Identify potential covert communication in AI-driven multi-agent systems by focusing on non-random numerical signal patterns inside transmitted messages.
Authors
Alessio Buscemi, Daniele Proverbio, Alessandro Di Stefano, The Anh Han, German Castignani, Pietro Liò
Abstract
Large language model (LLM)-based agents increasingly operate in multi-agent systems (MAS) characterised by strategic interaction. However, little is known about whether, and to what extent, different types of messages affect the outcomes of strategic games. By investigating AI agents based on four popular LLMs, playing four games with different cooperation equilibria, we study whether messages of different kinds (natural language, numerical signals, or random sequences) significantly modify the levels of cooperation in each game, also depending on the agents' assigned personalities. We observe that structured messages alter the final payoffs for most games and LLMs, but without a predictable pattern; this challenges the assumption that AI agents can converge to stable equilibria regardless of additional capabilities. Moreover, we observe that agent-generated numerical messages depart from randomness, most strongly and consistently when agents are explicitly instructed to communicate; however, they introduce an additional interpretability challenge, as their symbol distributions are mostly associated with the payoff structure and typically become more concentrated with repetition, but are overall difficult for humans to interpret. Monitoring for coordination of AI agents through restricted channels should thus prioritise message-level fingerprints, which generalise across models, over behavioural decisions, which do not.