Human-guided AI improves safety of medical image analysis
CHILD: Human-in-the-Loop OOD Detection for Safe Clinical Deployment
Computer Vision and Pattern Recognition
Summary
AI tools in medicine need to recognize when they see something unfamiliar to avoid making mistakes, but fully automatic adjustments can be risky without human help. The authors propose a method called CHILD that uses limited input from doctors to check only the most uncertain cases. This approach helps the AI better decide when it might be wrong without retraining the system. Tests showed that even with very little human feedback, CHILD made the AI more reliable at spotting unusual medical images.
Out-of-distribution detectionMedical AITest-time adaptationHuman-in-the-loopRisk-aware sample selectionScore calibrationFeature cacheAUROCFalse positive rateClinical deployment
Authors
Jinlun Ye, Kaiyue Lu, Runhe Lai, Xinhua Lu, Jia-Xin Zhuang, Ruixuan Wang
Abstract
Out-of-distribution (OOD) detection is critical for safe deployment of medical AI systems. Recently, test-time adaptation (TTA) has emerged as a new paradigm for OOD detection, automatically adjusting detector behavior during deployment. However, such automatic adaptation mechanisms may raise safety concerns in safety-critical clinical environments. While physician oversight can mitigate these risks, it is resource-intensive and must be judiciously allocated. To reconcile safety with efficiency, we propose CHILD, a training-free framework designed to enhance streaming OOD detection via sparse human feedback. Operating under strict budget constraints, CHILD employs an adaptive risk-aware sample selection mechanism to pinpoint only the most decision-uncertain samples for review. Crucially, it maximizes the utility of this sparse feedback through a retrieval-based score calibration module, which refines model predictions using a compact feature cache without any parameter updates. Extensive experiments on four medical benchmarks demonstrate that CHILD turns limited supervision into significant reliability gains: with a sparse feedback budget of only 5%, it reduces the average FPR95 from 72.63% to 60.26% and improves AUROC from 75.53% to 81.85%, consistently outperforming state-of-the-art baselines. Our code is publicly available at https://github.com/figec/CHILD.