Post-processing fairness fixes sometimes help but often mislead in ml models
When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations
Machine LearningComputers and Society
Summary
Fairness checks for machine learning models usually happen only once when the model is first used, but fairness can change over time or across different groups. The authors studied a common fairness adjustment method called ThresholdOptimizer across eight different areas like criminal justice and healthcare. They found that this method helped reduce unfairness when the original model was very biased, but often made things worse when the model was almost fair already. The study shows that a single fairness check at deployment isn't enough; ongoing checks and careful initial testing are needed.
What this means in practice
- •For machine learning engineers: Screen models for baseline unfairness before applying post-processing fairness fixes to ensure they reduce bias rather than increase it.
- •For ml operations teams: Use continuous monitoring tools like CUSUM to detect fairness regressions after deploying constrained models, enabling timely interventions.
Authors
Nithin Raghava Ramachandra Narla
Abstract
Fairness audits in production ML typically occur once, at deployment, on a single domain. Both fail in practice: fairness can shift after retraining or a changing user base, and interventions validated on one dataset are rarely tested across the heterogeneous domains an organization deploys. We present FAPE (Fairness Auditing for Production Environments), a four-stage framework evaluating a single post-processing intervention, Fairlearn's ThresholdOptimizer, across eight domain evaluations: criminal justice, income prediction, legal admissions, credit lending, agricultural lending, a multi-domain benchmark corpus, healthcare, and education. Each is scored on demographic parity and equalized odds difference, plus disparate impact ratio and accuracy cost where computable. Intervention effectiveness tracks baseline disparity magnitude: across model-domain pairs the constraint improved disparity in 9 of 14 high-disparity cases and worsened it in 3 of 4 near-fair ones. Each of the five high-disparity exceptions reverses under one of two measurement checks, a minimum group size or thresholds fit on held-out data. A CUSUM monitor started at deployment, tested on a simulated shift, separates constrained models that never met a 0.1 parity convention from those that met it and later regressed. A single deployment-time audit is therefore an unreliable guide, which argues for baseline-disparity screening and continuous monitoring