Fairness evolves with competing goals in reinforcement learning games
Evolution of fairness in multi-objective reinforcement learning framework
Machine Learning
Summary
Fairness is a tricky idea to explain because people care about more than just winning or getting the most money. The researchers built a computer model where fairness and getting rewards both matter, and they change how much each counts. When fairness was a bit important, people accepted unfair offers more often, which is something we actually see in real life. Their model shows how fairness can emerge from the balance between wanting rewards and behaving morally.
What this means in practice
- •For game developers: Model player behavior where decisions balance between personal gain and fairness to create more realistic social interactions in games.
- •For social robotics engineers: Design robots that can negotiate fairly by balancing reward objectives and fairness preferences in decision-making algorithms.
Authors
Jingyi Zhang, Xin Ou, Guozhong Zheng, Shengfeng Deng, Jiqiang Zhang, Li Chen
Abstract
Fairness, as a fundamental social norm, continues to pose a longstanding puzzle regarding its emergence. Traditional game-theoretic models largely rely on the assumption of \emph{Homo economicus}, wherein individuals are purely rational and self-interested, acting solely to maximize material payoffs. Such accounts, however, overlook the multidimensional nature of human decision-making, which is often shaped also by other considerations beyond economic incentives. To address this gap, we propose a multi-objective reinforcement learning framework that models the evolution of fairness as a dynamic trade-off between material payoff maximization and fairness-driven moral behavior, regulated by a fairness pressure coefficient. Using simulations of a two-objective Q-learning ultimatum game, we find that increased fairness pressure promotes fair outcomes, as expected. Strikingly, however, under moderate pressure, responder behavior reverses: responders become ``forgiving" by accepting low offers -- a pattern in line with our daily experience. Microscopic analyses reveal that this strategy reversal stems from competition between payoff-maximizing and fairness-oriented preferences. We further extend our framework to an asymmetric setting, where proposers and responders assign different weights to the two objectives. Overall, our work expands the reinforcement learning paradigm from a single-objective to a multi-objective formulation, offering a versatile tool for elucidating a broader range of human social behaviors.