AI summaryⓘ
The authors created Social Gym, a collection of 21 social games with clear rules and results designed to fairly test how well AI language models can interact with others. They found that while some models did well overall, none were perfect at every game or role, showing social skills remain challenging for AI. To help models improve without extra training, the authors invented SPaRTan, where a model plays, thinks about its moves and outcomes, then uses that insight to do better next time. This method improved some models in weaker roles but didn’t help all AI equally. Together, these tools offer a way to measure and boost AI social abilities in a clear and repeatable way without changing the model itself.
Large Language Models (LLMs)Multi-agent systemsSocial reasoningSelf-playReflectionTransfer learningElo ratingSocial gamesBenchmarkingModel evaluation
Abstract
LLM agents are increasingly deployed in multi-agent social settings where they must cooperate, negotiate, and adapt to other agents. Measuring and improving these social skills is hard because, unlike math or logic, social interaction offers no objective ground truth: evaluations fall back on LLM judges, which are costly, subjective, and noisy, and models get no reliable signal to learn from. To address both, we first introduce Social Gym, an environment of 21 multi-agent social games (e.g., Werewolves, Resistance, Spyfall) whose rule-decided outcomes make agent performance verifiable and objective, with an Elo tournament that produces a cross-game leaderboard. Benchmarking experiments show that while GPT-5-mini tops the leaderboard, no model excels at all games uniformly or in all game roles, pointing to limitations of social reasoning. Motivated by this, we additionally propose SPaRTan (Self-Play and Reflect-Transfer), a training-free self-improvement loop: a model plays a game, reflects on its trajectories and their outcomes to produce a transferable playbook, and applies that playbook in subsequent games. Our results show that SPaRTan playbooks help GPT-5-mini agents level their performance on weaker roles, but largely do not improve Qwen3-32B's performance. Together, Social Gym and SPaRTan offer a reproducible, verifiable foundation for measuring and improving LLM social reasoning without weight updates.