Fictitious play convergence studied beyond standard expected utility maximizers
Fictitious Play When Agents Are Not Expected-Utility Maximizers: A Geometric Analysis in $2$-Action Games
Computer Science and Game Theory
Summary
Fictitious play is a way for players in a game to learn by guessing what their opponents will do and then choosing the best response. Usually, it assumes players want to maximize their average payoff. This paper looks at what happens if players evaluate payoffs in more complicated ways, like being cautious about risks. The authors find some conditions that allow learning to still work well in simple two-action games and show it can fail in more complex games. Their work helps understand how different attitudes toward risk affect game learning.
What this means in practice
- •For game designers: Analyze player behavior in games where players may not maximize average outcomes but use risk-aware strategies, improving game balance and learning prediction.
- •For automated trading teams: Improve models for agents adjusting strategies when opponents use risk-sensitive decision rules, potentially enhancing adaptive trading algorithms.
A theory result. No direct application yet.
Authors
Hao Liang, Solaleh Mohammadi, Kaiqing Zhang
Abstract
Fictitious play (FP) is one of the earliest and most fundamental learning procedures in games, where each agent independently chooses a best response to the empirical distribution of opponents' past actions by maximizing the expected payoff with respect to it. However, learning agents may not always be expected-utility maximizers: they may use nonlinear evaluation functionals to account for the uncertainty induced by the opponents, e.g., via risk measures. In this work, we study nonlinear fictitious play (NL-FP), in which each pure action is evaluated by a general, possibly nonlinear functional of the payoff distribution induced by the opponent's empirical belief. To analyze NL-FP, we first identify a single-crossing property (SCP) of evaluation functionals that preserves the classical geometry of FP in $2\times2$ games. Furthermore, we conduct a novel geometric analysis of the convergence behavior of NL-FP in $2\times2$ games, even when the SCP fails. Beyond $2\times2$ games, we show that the convergence of NL-FP in $2\times n$ ($n\ge3$) games may fail even for continuous-time NL-FP, when the two-action player uses conditional value-at-risk (CVaR) in a $2\times3$ game. In addition, we identify a finite-crossing condition that ensures convergence of the selected continuous-time dynamics and the discrete-time iterates in general $2\times n$ games, which also provides new insights into the global landscape of continuous-time classical FP in such games.