The authors study how to make prediction intervals more reliable by ensuring that predictions are unbiased when looking at their own values, a property called self-calibration, and that intervals have correct coverage based on the prediction itself. They note an existing method achieves these goals but is very slow when outcomes are continuous. To fix this, the authors propose a new approach called Isotonic Conformal Prediction, which separates calibration from interval construction to improve efficiency. They introduce two versions: one that is fast and asymptotically self-calibrated, and another that achieves exact goals but is more computationally involved. Both new methods perform well on test problems and real data, matching accuracy of the slower method but at much lower computational cost.
Authors
Daniel Bensimon, Sean Xiang Yu, Eric D. Kolaczyk, Archer Y. Yang
Abstract
A point prediction that is well calibrated on average can still be systematically biased conditional on its own value, undermining its use in downstream decision-making. We consider two objectives for reliable uncertainty quantification: self-calibration, requiring a point prediction to be unbiased conditional on its own value, and prediction-conditional validity, requiring a prediction interval to attain nominal coverage conditional on the prediction. Self-Calibrating Conformal Prediction (SC-CP) attains both objectives exactly in finite samples, but requires refitting its calibrator for every candidate outcome, which is computationally prohibitive for continuous outcomes. We propose Isotonic Conformal Prediction (ICP), a framework that decouples calibration from prediction-set construction by fitting a single isotonic recalibration map and constructing prediction intervals within strata of similar recalibrated predictions. Within this framework we develop two procedures. Split Isotonic Conformal Prediction (SICP) attains prediction-conditional validity in finite samples and self-calibration asymptotically, at the computational cost of split conformal prediction. Transductive Isotonic Conformal Prediction (TICP) attains both objectives exactly in finite samples through a per-test-point inner loop that avoids refitting the isotonic calibrator. On synthetic heteroscedastic regression problems and a real-world healthcare-utilization dataset, both procedures match the coverage of SC-CP at substantially lower computational cost.