Learning algorithms shape game outcomes and enable collusion

How Learning Selects Equilibria and When It Leads to Collusion

Computer Science and Game Theory

Summary

This paper looks at how the design of learning algorithms affects which solutions players settle on in repeated games where they make decisions over time. The authors find that certain learning behaviors tend to select outcomes that benefit all players, sometimes leading to collusion where companies earn more than usual competitive profits. For example, in repeated market competition models, the learning process can push firms toward agreements that resemble monopoly profits rather than standard competition. These findings highlight how algorithmic learning can influence economic outcomes and explain why collusion might emerge without explicit agreements.

What this means in practice

  • For market designers: Recognize how learning dynamics among competing firms can lead to unintentional collusion beyond classical equilibria, informing regulation and market monitoring.
  • For multiagent system developers: Design learning algorithms for agents that influence which equilibria arise in repeated strategic interactions, tailoring cooperation or competition outcomes.

A theory result. No direct application yet.

Authors

Hanzhang Wang, Manxi Wu

Abstract

We study how learning algorithm design shapes equilibrium selection in infinite-horizon discounted Markov games with finite states and actions. Each episode, players choose smooth best responses to beliefs about opponents' policies, test them using noisy observations, and resample rejected beliefs or explore at rates determined by increasing transformations of their discounted values. We show that stochastically stable states induce approximate stationary Nash equilibria maximizing the minimum transformed discounted value across players, with transformations governing which players selection favors. In repeated games with bounded recall of noisy public signals, longer recall weakly increases the selected minimum transformed value. In repeated Cournot games, every stochastically stable state induces an approximate public perfect equilibrium whose minimum discounted profit across firms exceeds the Cournot-Nash benchmark. Collusion is therefore inevitable under these conditions and learning dynamics. For symmetric firms, selected profits can approach the monopoly benchmark arbitrarily closely. These results offer a novel equilibrium-selection perspective on algorithmic collusion.