Nonasymptotic error bounds for conformalized quantile regression under covariate shift
Conformalized Quantile Regression and Minimax Limits of Fixed-Score Calibration under Known Covariate Shift
Machine Learning
Summary
Predicting accurate ranges for future events based on data is tricky when conditions change between training and testing. The authors studied how well a method called conformalized quantile regression works when the data used to adjust the method differs from the data where predictions happen. They provided detailed mathematical limits on the errors in these predictions, even when relying on neural networks. Their work helps understand how reliable these prediction intervals are under different conditions.
What this means in practice
- •For machine learning engineers: Design prediction interval methods that remain accurate when training and test data distributions differ due to covariate shift.
- •For data scientists in finance: Build better risk assessment models by quantifying prediction uncertainty with guaranteed error bounds using neural network quantile regression.
Authors
Rustam Isaev, Anton Conrad, Denis Belomestny, Eric Moulines, Sergey Samsonov
Abstract
In this paper, we study nonasymptotic $L^p$ error bounds for interval length and conditional coverage in split conformalized quantile regression (CQR). Our bounds rely on local regularity conditions and accuracy guarantees for the estimated quantiles. We further instantiate our bounds for quantile regression with sparse ReLU neural networks. We also consider covariate shift, where the calibration and test covariates have different distributions, and derive nonasymptotic bounds for this setting. We obtain matching minimax upper and lower bounds in expectation for two constructed fixed-score calibration benchmarks under known covariate shift. The bounds match for every $p\in[1,\infty]$ in the scalar problem and for finite $p$ in the $K$-threshold problem; for the latter, a high-probability minimax lower bound holds for every $p\in[1,\infty]$.