Fairness measured by stability under changing group data
A Comprehensive View of Fairness through Distributional Stability
Machine LearningArtificial Intelligence
Summary
Deciding if computer predictions are fair is tricky because real-world groups can change in makeup. This paper looks at fairness as how much a prediction stays consistent when the group data shifts. The authors show that many common fairness ideas fit this view and create a way to measure unfairness using mathematical tools. They propose a method to train fair prediction systems that work well even as group compositions vary. Tests on common examples show how this approach can be applied.
What this means in practice
- •For machine learning engineers: Create prediction models that maintain fairness despite changes in demographic group distributions during deployment.
- •For product managers in ai companies: Develop fair AI products confidently without needing exact knowledge of future user demographics to guide fairness guarantees.$Commercial implications: Enables building fair AI tools with robust fairness across uncertain user groups, improving trust and compliance.
Authors
Gayane Taturyan, Charlotte Laclau, Stephan Clémencon
Abstract
We view fairness as a property of distributional stability. Rather than assessing a predictor under a fixed data distribution, we study how its predictions change under perturbations that modify the composition of protected groups. A predictor is fair if it remains stable under such shifts. Under this perspective, several classical notions of fairness arise as stability with respect to specific perturbations, with the associated unfairness gap given by a Lipschitz constant of a prediction-rate functional. This formulation also yields guarantees that hold uniformly over a range of demographic compositions at test time, without requiring knowledge of the deployment distribution. It leads to a learning procedure based on convex combinations of reweighted predictors, formulated as a second-order cone program, for which we establish generalization bounds. Experiments on standard benchmarks illustrate the approach.