Conformal calibration transfer improves prediction confidence across data shifts
Conformal Calibration Transfer
Machine Learning
Summary
Prediction models often need to say how confident they are in their guesses, but they can struggle when the data changes from where they learned. The authors study how to transfer calibration—methods that adjust confidence—from one data type (source) to another related but different type (target). They create a method that uses pairs of related unlabeled data to move calibration over, then fixes any leftover differences using only unlabeled information. Their approach helps make reliable predictions even when new labeled data isn’t available, and can signal when corrections are needed.
What this means in practice
- •For machine learning engineers: Generate reliable prediction sets on new sensors or input types without requiring new labeled calibration data.
- •For remote sensing analysts: Maintain trustworthy uncertainty quantification when switching between different satellite imaging modalities using only paired unlabeled data.
Authors
Achref Doula
Abstract
Conformal prediction converts point predictions into set-valued predictions with coverage guarantees under exchangeability between calibration and deployment data. We study conformal calibration transfer, where this requirement fails because labeled calibration is available only in a source space, while prediction sets are needed in a target space linked to the source through unlabeled paired observations (e.g., paired modalities or sensor changes). We propose Transported Conformal Calibration (TCC): we transport labeled source calibration into the target space using the paired data, and then correct residual post-transport mismatch using only unlabeled target inputs. We instantiate this correction with two complementary methods: TCC-KS, which uses a label-free uncertainty surrogate to detect mismatch and adjust calibration conservatively, and weighted-TCC, which reweights transported calibration toward the target domain for improved efficiency when weights are stable. We provide finite-sample target-domain coverage guarantees that adapt to an observable measure of mismatch. Across CIFAR-100-C, Tiny-ImageNet-C, and SEN12MS, we show reliable target-domain coverage transfer without labeled target calibration data, with label-free diagnostics that predict when correction is needed.