Symmetric strategies solve repeated harmony game using Bellman equation
Symmetric solution of the Bellman optimality equation for repeated harmony game
Computer Science and Game TheoryArtificial IntelligenceMachine Learning
Summary
Social dilemma games study how people cooperate or compete when rewards or punishments are added. This paper looks at a specific type of repeated game called a harmony game and finds three main types of cooperative strategies that can emerge. One strategy matches always cooperating, another matches a known win-stay lose-shift behavior, and the third is more complex. The authors also explore which of these strategies can be learned by artificial agents using reinforcement learning.
What this means in practice
- •For game designers: Develop repeated game strategies that promote cooperation by using symmetric solutions of the Bellman equation in harmony-like scenarios.
- •For reinforcement learning engineers: Train agents to discover cooperative strategies in social dilemmas by applying the identified symmetric solutions as optimal behaviors.
Tested on simulated data.
Authors
Hisato Komatsu
Abstract
In social dilemma games, additional rewards or punishments have been studied as means of promoting cooperation. Therefore, it is important to investigate the ideal situation, in which such an additional payoff would change the game. In this study, we investigated the symmetric solution of the Bellman optimality equation for a repeated harmony game. The calculations showed that three types of symmetric solutions exist. One of them corresponds to the trivial All-C strategy, and another to the Win-stay Lose-shift strategy of the prisoners dilemma game. The nontrivial behavior of the strategy corresponding to the last solution is also discussed in detail. In addition, we numerically investigated which strategy the agents actually learn by the reinforcement learning algorithm.