Papers for

algorithms developers for fairness

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Omniprediction does not require fairness but loss indistinguishability does

Multigroup Fairness and Omniprediction: Separations and Equivalences

Abstract: Omniprediction is a learning guarantee which requires a single predictor to be competitive relative to the best hypothesis from a benchmark class for any loss chosen from a family of loss functions. Loss Outcome Indistinguishability (loss OI for short) is a stronger notion that implies omniprediction. It requires the predicted distribution on labels to be indistinguishable from the true distribution to tests that depend on the loss functions and the benchmark class. Multiaccuracy and multicalibration are multigroup fairness notions that generalize classical notions of calibration and accuracy in expectation. Most known learning algorithms for omniprediction (both for the standard notion and for strengthenings like loss OI) rely on some version of these multigroup fairness notions, or on an intermediate notion called calibrated multiaccuracy. We ask if this is necessary: Does omniprediction require some form of multigroup fairness? We show that the answer is no for (plain) omniprediction, and yes for loss OI. First, a sequence of works shows that multicalibration or calibrated multiaccuracy imply omniprediction. We rule out even a weak converse, by showing that omniprediction for proper losses does not imply even accuracy in expectation, a much weaker notion than any of calibration, multiaccuracy, or multicalibration. Second, prior work showed how to achieve loss OI from a combination of calibration and multiaccuracy. We show a converse: loss OI is equivalent to a form of calibrated multiaccuracy.

Mon 5 OctMachine Learning
The gist
This paper looks at how fairness concepts connect to omniprediction, a way to make predictions that work well across many different types of errors. The authors find that for basic omniprediction, fairness ideas like being accurate on many groups are not needed. However, for a stronger goal called loss outcome indistinguishability, a form of fairness is necessary. They clarify that while fairness helps achieve these stronger guarantees, it is not required for simpler prediction goals.
Open → 2610.07374v1