How learning agents develop bias and polarization in groups
Emergent Discrimination and Polarization in Multi-Agent Systems
Multiagent Systems
Summary
This paper looks at how groups of learning agents can start to form biases and opposing opinions on their own. The agents update what they believe and who they trust by interacting with one another and learning from these conversations. When agents are labeled into two groups with some bias, the whole system can settle into different patterns of opinion and trust, showing various types of discrimination and polarization. The authors use ideas from physics to connect these local updates to large-scale behaviors, which could help us study similar phenomena in human societies.
What this means in practice
- •For multi-agent system designers: Evaluate and predict group-level polarization and bias in systems where agents learn and interact over time, improving safety measures.
- •For social network platform engineers: Identify measurable indicators to monitor and understand collective opinion formation and discrimination patterns in user interactions.
Authors
Nestor Caticha, João Henrique Sant'Ana, Felippe Alves, Davi Bastos Costa, Mariana Mercucci, Juan P. Neirotti, Otavio Citton, Renato Vicente
Abstract
We study theoretically how discrimination and polarization can emerge in a population of learning agents. Each agent uses an adaptive linear decision layer to map a language-model embedding of an issue to a binary opinion and maintains an evolving estimate of how much it trusts every other agent. Through pairwise interactions, agents update both their decision weights and trust estimates using an online Bayesian rule derived from teacher-student learning over a noisy channel. In-context-learning experiments with large language models qualitatively reproduce the predicted opinion and trust updates. We then assign each agent one of two arbitrary group labels, A or B, and model group bias by allowing a fraction $f_b$ of agents to shift its perceived agreement with each message by a strength $b$, a bias steering prejudice, depending on whether the sender and receiver share a label. We use order parameters from statistical mechanics to connect these local dynamics to collective states and transitions. Starting from uniformly random initial conditions, simulations reveal a rich phase diagram in the $(b,f_b)$ plane, comprising four collective states: spin-glass frustrated out-group favoritism, nondiscriminatory polarization, discrimination with opinion alignment, and discrimination without opinion alignment. Which state emerges depends on both the strength and prevalence of the bias. These results show how simple local learning dynamics can generate qualitatively distinct forms of polarization and discrimination, underscoring the need for population-level safety analysis of multi-agent systems. They also suggest measurable order parameters for characterizing analogous collective states in human societies.