Study links shortcut learning in AI to calibration issues and offers fixes
Prevalence calibration as shortcut mitigation
Machine LearningComputer Vision and Pattern Recognition
Summary
Sometimes AI systems make decisions based on misleading clues called shortcuts instead of important information. This happens because the AI learns to trust these shortcuts based on how common they were in the training data. The authors show that this problem is related to how confident the AI is about different groups of data, and they propose ways to adjust this confidence to reduce shortcut use. Their methods work well on medical imaging tests and can improve AI even when starting from powerful pretrained models. This connects understanding shortcuts with ideas from fairness and confidence in AI predictions.
Shortcut learningCalibrationSpurious correlationsFoundation modelsPost-hoc recalibrationChest drain pneumothoraxCheXpertSIIM-ACRAlgorithmic fairnessAUROC
Authors
Mohamed Amine Kina, Eike Petersen
Abstract
Shortcut learning denotes the widespread situation in which a classifier exploits spurious correlations rather than diagnostic features. Existing mitigation strategies mostly aim to learn shortcut-invariant representations; their empirical success is limited and they cannot be applied to classifiers using frozen foundation model encoders. We propose to reframe shortcut learning as fundamentally a calibration problem: unconstrained learning implicitly calibrates each shortcut group to its training set disease prevalence, rendering the resulting classifier necessarily over-confident in one group and under-confident in the other. Building on this insight, we prevalence-equalize calibration between shortcut groups through two encoder-agnostic methods, an in-processing regularizer and a post-hoc prevalence-equalized recalibration step. Across chest-drain-pneumothorax benchmarks on CheXpert and SIIM-ACR, spanning fine-tuned CNNs and frozen foundation-model backbones, both methods substantially outperform all baselines. Post-hoc recalibration of a standard ERM-trained DenseNet raises misaligned-group AUROC from 0.23 to 0.73, indicating that shortcut reliance degrades the classification head rather than the underlying representation. Besides two new state-of-the-art shortcut mitigation approaches, our findings more fundamentally connect shortcut learning to calibration theory and algorithmic fairness.