Summary
Medical imaging tools often guess parts of an image, like blood vessels or skin lesions, but these guesses can be uncertain. The authors show that common ways to measure this uncertainty don’t tell us if it actually helps doctors make better decisions when reviewing images. They created a new way to test uncertainty that looks at how well it helps decide which parts need closer review without wasting time. Their tests across different medical images and methods show that the best uncertainty approach depends on how review is done and what costs are involved. This means it’s better to think about uncertainty together with how images are reviewed, rather than alone.
What this means in practice
- •For medical imaging teams: Optimize image review workflows by deciding which regions need human checking based on uncertainty to reduce errors within fixed time or budget limits.
- •For quality assurance teams: Develop better evaluation protocols for uncertainty measures to improve trust and efficiency in automated diagnostic systems.
Abstract
Medical segmentation uncertainty is often evaluated with calibration error or pixel-level error-detection scores, but these metrics do not show whether uncertainty improves downstream review decisions. We propose a decision-aware evaluation protocol that separates uncertainty estimation from deferral policy and evaluates their combination under fixed review budgets: which pixels, regions, or images should be accepted, and which should be reviewed. The protocol reports fixed-budget error reduction, risk-coverage behaviour, region-level review proxies, structural failure modes, probability-derived baselines, and sensitivity to review assumptions. We instantiate it across retinal vessel, skin lesion, and polyp segmentation with multiple backbones and uncertainty estimators. Under matched inference budgets and canonical multi-layer MC Dropout, test-time augmentation (TTA) gives a stronger error-detection signal across matched MC--TTA comparisons, including architecture and dropout-placement checks. However, estimator rankings are not universal. For example, probability-derived scores remain competitive, deep ensembles can outperform TTA, learned deferral helps mainly when the base estimator is weak, and rankings can change with review unit, uncertainty target, correction assumptions, and cost model. Structural and second-observer analyses further show that boundary and thin-structure failures are partly tied to annotation ambiguity. These findings support evaluating uncertainty as an estimator-policy-review configuration, not as a standalone score map. Code: https://anonymous.4open.science/r/Uncertainity-in-Medical-Image-Segmentation/README.md