Counterfactual marginalisation helps evaluate robustness to bias
Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables
Machine LearningArtificial Intelligence
Summary
Machine learning models can mistakenly rely on things like a person’s age or sex to make decisions, which can cause errors. The authors propose a way to test models by changing these 'nuisance' factors in images and averaging the model’s predictions across these changes. This method helps reveal whether a model truly understands the important details or just uses shortcuts based on demographics. They also introduce measures to check how stable and reliable the model is under such changes.
What this means in practice
- •For medical ai developers: Evaluate and improve clinical image diagnosis models’ robustness by testing how predictions hold up when demographic factors are varied artificially.
- •For autonomous vehicle engineers: Assess vision system reliability under changes in environmental or acquisition-related variables using intervention-aware averaging techniques.
Authors
Yasin Ibrahim, Hermione Warr, Robin J. Evans, Konstantinos Kamnitsas
Abstract
Machine learning models can achieve strong test performance while relying on demographic or acquisition-related shortcuts. We propose counterfactual (CF) marginalisation as a test-time evaluation procedure for assessing robustness of classification models to such variables. Given a CF image generator, we intervene on nuisance parent variables such as age or sex, generate CF versions of each test image, and average predictions over a target intervention distribution. This produces intervention-aware predictions that marginalise demographic effects while preserving patient-specific latent information. We use these predictions to define metrics for CF risk, calibration, stability and worst-case sensitivity. We demonstrate this framework's utility for quantitative robustness evaluation.