Large language models improve social skills through reinforcement fine-tuning
Playing social deduction games with reinforcement fine-tuned large language models
Computation and LanguageArtificial IntelligenceMultiagent Systems
Summary
Playing social deduction games is hard because players need to guess others' secret roles and influence votes. The authors studied how training large language models (LLMs) with reinforcement helps them get better at these social skills. They found that just winning games doesn't teach LLMs much, but training focused on understanding social clues and influencing decisions improves their behavior. Combining both social reading and influence training works best and makes LLMs more convincing and strategically smart according to human judges.
What this means in practice
- •For game ai developers: Build AI agents that understand hidden roles and influence player decisions in social deduction games.
- •For social robotics engineers: Create robots or agents better at reading social cues and guiding group decisions in interactive settings.
Authors
Lingzhe Zhang, Yunpeng Zhai, Tong Jia, Kening Zheng, Chiming Duan, Minghua He, Zhaoyang Liu, Bolin Ding, Philip S. Yu, Ying Li
Abstract
Reinforcement fine-tuning (RFT) is increasingly used in applications where large language models (LLMs) interact with humans and other agents. Here we use social deduction games to study how RFT changes LLMs' social behaviour. We let fine-tuned and base LLM agents play hidden-role games that require hidden-state inference, social reading and vote steering. Our results show that LLM agents do not reliably acquire social-deduction ability by directly optimizing terminal win--loss outcomes, suggesting that final game results provide a sparse and noisy signal for socially interactive learning. However, RFT is particularly effective at improving social reading, including tasks that require agents to infer hidden roles from public discussion, update beliefs over time and predict other agents' future decisions. We further show that RFT can also improve social influence, including tasks that require agents to steer votes, team approvals and collective decisions, although these gains depend more strongly on behaviourally specific rewards and structured interaction settings. Finally, we show that LLMs' ability to play social deduction games can be further improved through multi-agent social-cognitive reinforcement fine-tuning, which combines social-reading and social-influence signals during same-side multi-agent training. These learned behaviours also receive more favourable human evaluations of strategic competence, persuasiveness and social usefulness. Together, these results enrich our understanding of how RFT changes LLMs' social behaviour and provide a step toward a behavioural learning theory for machine social intelligence.