Llm agents learn weak cartel enforcement only when forced
Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets
Computation and LanguageComputer Science and Game TheoryMachine Learning
Summary
The paper looks at how language model agents behave in markets where they might try to cooperate to keep prices high, like a cartel. True cartels must punish members who cheat on the group. The authors create a test that forces an agent to cheat and see if others punish it. They find that these AI agents only learn to punish cheating if explicitly trained to do so, otherwise they mostly ignore rivals or match cheaters' price cuts. This explains why simple outcome measures can’t tell if agents really enforce cartels or just act like they do.
What this means in practice
- •For market design teams: Test automated pricing agents to detect whether they truly enforce collusive behavior or just mimic outcomes without punishment.
- •For ai safety engineers: Use enforcement audits to improve trustworthiness of multi-agent systems that interact competitively or cooperatively in economic settings.
Tested on simulated data.
Authors
Hieu Tran, Hong Yu
Abstract
Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading. Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts. Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.