Summary
As AI systems interact more on their own, understanding when they choose to cooperate is important. This paper studies how different AI models decide to cooperate or not in repeated games, by changing opponents' reputations, strategies, and emotional signals. Simpler AI models respond to reputation, strategy, and emotions similarly to humans, but they weigh these factors differently. More advanced reasoning models rely more on strategy and reputation, with less influence from emotions, and show varied behaviors at the end of interactions. These insights help understand how AI might work in real-world negotiations and social tasks.
What this means in practice
- •For negotiation system developers: Adjust AI negotiation agents to account for strategy and reputation effects in repeated interactions for more reliable cooperation.
- •For customer service platform teams: Design AI agents that interpret user reputation and emotional cues to improve cooperative responses in multi-turn conversations.
Authors
Celso de Melo, Zishan Feng, James Hale, Kazunori Terada, Giorgio Coricelli, Jonathan Gratch
Abstract
As generative AI (Gen AI) systems take on increasingly autonomous roles in economically and socially consequential interactions, understanding their propensity to cooperate -- and the signals that shape this propensity -- has become essential. We examine cooperative behavior in frontier Gen AI models using the iterated prisoner's dilemma, manipulating counterpart reputation (positive, unknown, negative), strategy (extortion vs. generosity), and non-verbal emotional signaling (facial expressions conveying competitive or cooperative appraisals). In a first study with non-reasoning models (Claude 3.5, Gemini 2.0 Flash, GPT-4o), cooperation was systematically shaped by all three factors, paralleling patterns long documented in human behavioral research, though models varied substantially in how heavily each factor was weighted. A second study with reasoning models (Claude 4.6, Gemini 3, GPT-5.2) revealed a more concentrated reliance on strategy and reputation, a near-elimination of the Potemkin effect observed in non-reasoning models (evidenced by near-uniform cooperation in a diagnostic harmony game), and a more conditional role for emotion consistent with a hierarchical cue-weighting strategy rather than a simple loss of social sensitivity. Reasoning models also showed heterogeneous end-game behavior, ranging from sustained cooperation to systematic last-round defection effect, revealing model-specific exploitability profiles with direct practical relevance for deployment in negotiation and other multi-round interactions. Together, these findings characterize Gen AI models as increasingly sophisticated, though heterogeneous, social actors, and underscore the practical value of developing standardized cooperation benchmarks to inform the responsible deployment of Gen AI in interactive, socially consequential settings.