Tabular foundation models get more accurate confidence predictions
Conformal Prediction and Conditional Coverage for Tabular Foundation Models
Machine Learning
Summary
Predicting outcomes with confidence intervals can sometimes be inaccurate even when the main predictions are good. The authors present a method called C-USIM that adjusts these confidence intervals so they better match the true uncertainty, even when the prediction is complicated. This method doesn’t need extra training and works with multimodal predictions from tabular foundation models. Their tests show it improves how well the confidence predictions cover the true outcomes and reduces errors in group predictions.
What this means in practice
- •For data science teams: Provide more reliable and calibrated uncertainty estimates in tabular regression models by using the C-USIM method without retraining the model.
- •For healthcare analytics teams: Improve confidence assessments in patient outcome predictions from tabular foundation models, enhancing trust in AI-assisted medical decisions.
Authors
Sungwoo Park, Sunghee Park, Won Chang
Abstract
Tabular foundation models (TFMs) provide predictive distributions for regression, but their prediction regions can exhibit undercoverage or overcoverage even when point predictions are accurate. We introduce C-USIM (Conditionally-Uniformized Score Integration Method), a lightweight application of highest predictive density split conformal prediction that accommodates multimodal predictions. Given calibration and test outputs, it requires no additional training or model inference. It provides finite-sample marginal validity under our assumptions. We bound conditional-marginal coverage gaps using distribution-estimation error and score discreteness, and examine coverage heterogeneity through percentile rank-score plots. Experiments with TabPFN and TabICL show improved marginal coverage accuracy and lower average conditional and group coverage errors. Under a fixed data budget, allocating more observations to calibration can reduce marginal coverage error despite less accurate point predictions.