Fair multi-armed bandit strategies minimize regret with new geometric measure
Nash Social Welfare for Multi Armed Bandits: Trajectory-wise Expected and High Probability Regret
Machine Learning
Summary
This work looks at how to fairly choose options in situations where you try different choices and learn which is best, called the multi-armed bandit problem. The authors found that previous measures of fairness didn’t fully account for outcomes over time. They propose new ways to measure fairness that better reflect the overall experiences across all trials and provide algorithms that perform optimally under these measures. Their approach also offers stronger guarantees on fairness and regret that hold with high probability, and they verified their results with simulations.
What this means in practice
- •For online platform operators: Improve fairness in recommendation systems by optimizing geometric mean reward over user interactions instead of average round returns.
- •For ad tech engineers: Design ad allocation algorithms that ensure fair distribution of rewards across multiple ad slots over time with strong probabilistic guarantees.
Authors
Avishek Ghosh
Abstract
We study fair multi-armed bandits under the Nash Social Welfare (NSW) objective, which measures performance via the geometric mean of accumulated rewards. Existing work defines Nash regret as $\mathrm{NR}_T = μ^\star - (\prod_{t=1}^T \mathbb{E}μ_{I_t})^{1/T}$, where $μ_{I_t}$ is the mean reward of the recommended arm $I_t$ and $T$ is the horizon. Since it applies the geometric mean to per-round marginal expectations, it ignores the joint distribution of rewards across rounds, leaving the NSW fairness motivation unaddressed at the trajectory level. We propose \emph{trajectory-wise Nash regret} $\widetilde{\mathrm{NR}}_T = μ^\star - \mathbb{E}[(\prod_{t=1}^T μ_{I_t})^{1/T}]$, which computes the geometric mean over complete sample paths before taking expectations, capturing NSW fairness more faithfully. By Jensen's inequality, $\widetilde{\mathrm{NR}}_T \geq \mathrm{NR}_T$, making it a strictly stronger metric. We also introduce \emph{high probability Nash regret} $\widehat{\mathrm{NR}}_T = μ^\star - (\prod_t μ_{I_t})^{1/T}$, giving the first high probability regret bounds in fair bandits. Our two-phase algorithm, Round Robin Nash Confidence Bound (\texttt{RR-NCB}), combines round robin exploration with a Nash confidence bound index policy. We show $\widetilde{\mathrm{NR}}_T \leq \widetilde{\mathcal{O}}(\sqrt{k\log T/T})$ and, with probability $1-δ$, $\widehat{\mathrm{NR}}_T \leq \widetilde{\mathcal{O}}(\sqrt{k\log(kT/δ)/T})$, matching the optimal $\widetilde{\mathcal{O}}(\sqrt{k/T})$ rate despite the stronger metrics. Optimality follows from a lower bound via AM-GM and standard $k$-armed bandit minimax arguments. Simulations validate our theory.