One example can greatly improve fairness test scores of AI models
One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs
Computation and LanguageArtificial Intelligence
Summary
Fairness tests help check if AI models treat people fairly, but this paper shows these tests may be too simple. The authors found that training or showing a large language model just one example from a fairness test can dramatically boost its performance on that test. This means the models might be using simple shortcuts rather than genuinely being fair. The paper suggests that fairness tests need to cover a wider range of challenges to better measure true fairness in AI.
What this means in practice
- •For machine learning engineers: Evaluate and improve AI fairness tests by incorporating diverse challenge types beyond simple multiple-choice benchmarks.
- •For ai ethics teams: Design more robust fairness evaluation protocols that avoid models exploiting simplistic cues to appear fair.
Authors
Naihao Deng, Samee Arif, Shuaichen Chang, Yulong Chen, Rada Mihalcea
Abstract
Warning: This submission studies stereotypes and biases, and contains toxic and offensive examples, used for illustration purposes only. Fairness benchmarks such as BBQ have become the de facto standard for fairness evaluation across major model families. We argue that these benchmarks are too easy to support their role: training Qwen 2.5 7B Base with Group Relative Policy Optimization (GRPO) on a single BBQ example, or placing that example in context as a one-shot demonstration for in-context learning (ICL), lifts mean BBQ accuracy from 79.9% to 92.9% and 99.0%, respectively, closing 80% of the gap to its large-scale RLHF counterpart (96.1%) with GRPO, and surpassing it with ICL. These effects generalize across model families. A cross-conditioning analysis shows the improvement is carried by the reasoning traces generated by the model, and one example suffices to elicit a category-agnostic ``missing evidence'' reasoning pattern. We argue that BBQ-style multiple-choice abstention benchmarks measure a single structural cue, and a model that solves them does not thereby become fair. We call for evaluation suites that cover a broader spectrum of fairness alignment.