Cooperation stability does not ensure agents can learn to coordinate
Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence
Artificial Intelligence
Summary
Cooperation in groups of independent agents is tricky because each agent adapts based on what others do. The authors show that even if a cooperative outcome is stable in theory, it may not be reachable by typical learning methods that agents use from limited experience. They tested this idea in a three-agent scenario involving a government, a firm, and users, finding that some learning algorithms failed to find cooperation despite the stability predicted by evolutionary models. This suggests that what’s stable in theory might not always be practical for learning agents.
What this means in practice
- •For multi-agent system designers: Design coordination strategies knowing that theoretical stability may not guarantee practical learning success among agents.
- •For platform governance engineers: Improve governance algorithms for shared resource platforms by accounting for differences between stable outcomes and what agents can learn.
Authors
Yijie Wang
Abstract
Cooperation emergence is a central problem in multi-agent systems because decentralized agents must coordinate while adapting to the changing behavior of others. Evolutionary game theory identifies strategically stable outcomes, but stability under a population adjustment dynamic need not imply that finite-sample learning agents can reach the same outcome through local reward feedback. We study this distinction in a transparent three-agent governance-motivated game involving a government, a platform firm, and users. We derive replicator dynamics for the fixed stage-game incentives, evaluate the cooperative evolutionary basin on a symmetric initial-condition grid, and compare it with learning-basin estimates for three decentralized value-based learners. The learning analysis uses independent Q-learning with $\varepsilon$-greedy action selection, scaled Boltzmann exploration, and SA--EA BQL under the same payoff environment and outcome criterion. The evolutionary basin has volume $V_E=1.00$ on the sampled grid. The empirical learning basin is $0.88$ for $\varepsilon$-IQL and $0.00$ for both scaled Boltzmann and SA--EA BQL. Diagnostic traces show that broader action diversity and nonzero value separation can coexist with failure to sustain the cooperative joint action in this fixed configuration. These results indicate that evolutionary stability and learning accessibility are distinct properties of a coupled game--learning system. The shared-bike setting is a motivating application; the broader contribution is a framework for comparing population-level stability with the finite-sample accessibility of cooperation under specified multi-agent learning dynamics.