Uncertainty of Vision Medical Foundation Models
2026-08-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied how well different AI models can estimate uncertainty when analyzing medical images like retinal scans and X-rays. They found that models trained specifically on medical data tend to be better at predicting how confident they should be than those trained on general images. They also showed that usual methods to adjust confidence scores are not enough to fix differences caused by training data. Finally, the authors demonstrated that specialized medical models can make more reliable prediction ranges, which is important for trustworthy AI in healthcare.
uncertainty estimationmachine learningconformal predictioncalibrationfoundation modelsself-supervised learningmedical imagingprediction setsdomain-specific training
Authors
Haoxu Huang, Narges Razavian
Abstract
Accurate uncertainty estimation is essential for machine learning systems de- ployed in high-stakes domains such as medicine. Traditional approaches primarily rely on probability outputs from trained models (point predictions), which provide no formal guarantees on prediction coverage and often require additional calibra- tion techniques to improve reliability. In contrast, conformal prediction (region prediction) offers a principled alternative by generating prediction sets with finite- sample validity guarantees, ensuring that the ground truth is contained within the set at a specified confidence level. In this study, we explore the impact of pre-training approach, dataset scale and domain on both point and region-level uncertainty quantification, by studying domain-specific vision medical foundation models vs. general domain vision foundation models. We conduct a comprehensive evaluation across foundation models trained on retinal, histopathological, and Chest X-Rays data, applying various calibration techniques. Our results demonstrate that (1) pre-training on higher-quality domain-specific datasets along with self-supervised learning leads to better-calibrated point predictions than general domain pre-training, (2) stan- dard re-calibration methods alone cannot fully mitigate uncertainty discrepancies across models trained on different data sources, (3) domain-specific foundation model can lead to more efficient conformal prediction. These findings highlight the importance of careful model selection and the inte- gration of both point and region prediction to enhance the reliability and trust- worthiness of medical AI systems. Our work underscores the need for a holistic approach to uncertainty quantification in recent development of medical vision foundation model, ensuring robust and interpretable AI-driven decision-making.