Collective cooperation without individual fidelity in LLM agents
2026-06-29 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors tested how well large language models (LLMs) can mimic human decision-making in a social game called the Prisoner's Dilemma. They found that while LLMs captured the overall patterns of cooperation seen in humans, they did not replicate the variety and specific ways people decide to cooperate or not. Even adding random choices helped only a little, meaning LLMs sometimes act like groups of humans but not like individuals. The authors suggest that to trust LLMs as stand-ins for humans, we need to check not just the big picture but also individual behaviors and decision rules.
Large Language ModelsPrisoner's DilemmaCooperation DynamicsSocial SimulationAgent-based ModelingBehavioral HeterogeneityNetwork TopologyConditional CooperationEmpirical Benchmark
Authors
Henrique Ferraz de Arruda, Carlos Gracia Lázaro, Alberto Aleta, Yamir Moreno
Abstract
Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making. Here we test LLM agents against a direct empirical benchmark: a large-scale networked Prisoner's Dilemma experiment with human participants. Using the same interaction protocol, payoff structure, and network topologies, we compare nine open-weight LLMs with the human data. The selected model reproduces several macro-level features of cooperation dynamics, including the early decline and later stabilization of cooperation. This aggregate agreement, however, does not extend uniformly to finer levels of behavior. LLM populations underestimate individual-level heterogeneity and generate conditional cooperation patterns that differ from those observed in humans. Adding a fraction of random agents improves some aspects of micro-level agreement, but does not remove the mismatch in decision rules. These findings reveal a macro--micro dissociation in LLM-based social agents: collective outcomes can appear human-like even when the underlying behavioral distributions and mechanisms are not. They suggest that validating LLM agents as human surrogates requires comparisons across aggregate dynamics, individual heterogeneity, and context-dependent decision rules, rather than outcome-level agreement alone.