Learning strategies from preference comparisons in continuous games
Learning in Continuous Games from Pairwise Preference Feedback
Computer Science and Game TheoryMachine Learning
Summary
This paper looks at how players can learn to make better decisions in games where they only know which choice is preferred between two options, not how much better it is or the actual payoff values. The researchers found that traditional measures of learning success don’t work with just this kind of preference information. So they developed a new way to measure and improve learning based solely on these preference comparisons, leading to guaranteed improvements over time and better predictions of game outcomes. Their method can learn efficiently from very limited feedback, without needing to know exact scores or utilities.
What this means in practice
- •For online marketplaces: Improve recommendation systems using only user preference comparisons without requiring exact ratings or scores.
- •For game designers: Design adaptive game AI that learns player preferences through simple preference queries rather than explicit payoffs.
Authors
Anas Barakat
Abstract
We study learning in continuous games when players receive only pairwise preference feedback, revealing which of two actions is preferred but neither payoff values nor preference magnitudes. We first show that standard external regret and coarse correlated equilibria (CCE) are not identifiable from this ordinal information: the same sequence of play can incur zero and linear regret in two ordinally equivalent games, while the distributions that remain CCE across all cardinal representations consistent with the same preferences are exactly those supported on pure Nash equilibria. Motivated by this gap, we develop a first-order ordinal theory based on normalized unilateral preference directions, introducing an ordinal directional regret benchmark and corresponding equilibrium notions. We show that block-normalized pseudogradient dynamics achieve sublinear ordinal regret and, under additional structure, Nash-convergence guarantees. We then use a single-comparison estimator to implement these dynamics from finite pairwise comparisons. With one comparison per player and round, the resulting algorithm achieves sublinear finite-resolution ordinal regret against arbitrary opponent behavior and, in ordinal potential games, almost-sure last-iterate convergence to the Nash set. Our results provide regret, dynamics, and equilibrium guarantees directly from preference feedback without reconstructing cardinal utilities.