Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning
Machine LearningArtificial IntelligenceCryptography and SecurityComputers and Society
Summary
The authors study how fairness works when machine learning systems use differential privacy (DP) to protect user data. They argue that it's not enough to look at fairness only in terms of accuracy outcomes; how much privacy risk each demographic group carries matters too. They introduce a new fairness measure called the Privacy-Cost Equity Ratio (PCER), which compares the benefits a group gets to how much privacy exposure it suffers. Their tests show that existing fairness checks can miss important issues, like when a group faces both worse predictions and higher privacy risks. They suggest fairness evaluations should consider both who benefits and who bears the privacy costs.
Authors
Rakshit Naidu
Abstract
Differential privacy (DP) is increasingly deployed to limit membership inference risk in machine-learning systems. Prior work has shown that DP-SGD can widen accuracy disparities across demographic groups, but this framing treats fairness as a purely outcome-side concern. We argue that privacy cost, the information leakage borne by each group, is itself a form of harm, and adopt a compensatory-fairness framework in which a group that involuntarily bears greater privacy exposure is owed proportionally greater benefit from the system. From this principle we derive the \emph{Privacy-Cost Equity Ratio} (PCER), a group fairness metric defined as a group's positive prediction rate normalized by its per-group overfitting gap. By a standard membership inference bound, this overfitting gap upper-bounds each group's vulnerability to inference attacks, making PCER a conservative measure of benefit relative to exposure. PCER needs only per-group train and test accuracy (no shadow models), making it a practical post-hoc audit tool. We evaluate PCER alongside standard fairness metrics across six benchmark--attribute combinations spanning tabular and NLP domains, under DP-SGD at a range of privacy budgets, and validate the overfitting-gap proxy against a direct threshold membership-inference attack. The results reveal patterns that outcome-based metrics miss. On COMPAS, PCER uncovers a persistent double disadvantage: the protected group bears both greater privacy exposure and worse predictive outcomes, something demographic parity gap masks entirely. Sensitivity analysis shows very strong privacy guarantees collapse both groups' overfitting to a numerical floor, rendering exposure-based audits uninformative in that regime. Together, these findings show that fairness audits of privacy-preserving systems must account for who bears the cost of protection, not only who benefits from its outcomes.