Informative missing labels can change classification results depending on goals
Decision-Target Reversal under Informative Label Missingness
Machine Learning
Summary
Sometimes when labels are missing in training data, the pattern of what is missing can actually help make better predictions. The authors found that whether this hidden information helps or hurts depends on what kind of mistakes are more costly when using the model. They call this change decision-target reversal, meaning a model that looks good under one goal can look worse under another. They explain this with math and examples, showing that the benefit of missing labels depends on both the data and the practical task.
What this means in practice
- •For machine learning engineers: Improve model deployment decisions by accounting for how label missingness patterns affect cost-sensitive classification outcomes.
- •For medical data analysts: Adjust diagnostic models for clinical tasks by recognizing that label availability patterns can influence performance under different medical cost priorities.
Tested on simulated data.
Authors
Farinborz Setoudehtazangi, Geoffrey J. McLachlan
Abstract
Informative label missingness can improve classification efficiency because the pattern of label availability may itself contain information about the discriminant structure. We show that this advantage need not persist across deployment objectives: with the data-generating model and statistical experiments held fixed, informative partial classification can be favourable under one loss and unfavourable under another. We call this phenomenon decision-target reversal. For binary Gaussian discrimination, we derive a local cost-sensitive excess-risk expansion and, in one dimension, obtain explicit critical cost ratios at which the first-order risk ordering reverses. We show that the required covariance geometry arises under finite uncertainty-dependent missingness mechanisms with a fixed marginal missing-label proportion. We then extend the analysis to multiclass finite-action problems through loss-specific active-boundary curvature, yielding a general decision-target frontier. Monte Carlo experiments illustrate the predicted reversals, and a descriptive gastrointestinal-lesion application with agreement-defined label availability illustrates sensitivity to deployment costs. The results show that favourable informative missingness is a property of the statistical experiment relative to the deployment decision, rather than of the experiment alone.