Human feedback improves AI safety in clinical out-of-distribution detection
CHILD: Human-in-the-Loop OOD Detection for Safe Clinical Deployment
Computer Vision and Pattern Recognition
Summary
AI systems in medicine can make mistakes when they see unusual cases different from what they were trained on. The authors developed CHILD, a system that asks doctors to check only the most uncertain cases during use, saving time. CHILD then uses those few expert inputs to improve its predictions without needing complex retraining. This approach helps make AI tools more reliable and safer for clinical use.
What this means in practice
- •For hospital data teams: Improve AI diagnostic reliability by selectively involving clinicians to review uncertain cases in real-time AI deployment.
- •For medical ai product developers: Develop safer clinical AI tools that adapt predictions on-the-fly using minimal expert feedback without retraining models.$Commercial implications: Enables creation of adaptive clinical AI products that reduce incorrect alerts using sparse physician supervision.
Authors
Jinlun Ye, Kaiyue Lu, Runhe Lai, Xinhua Lu, Jia-Xin Zhuang, Ruixuan Wang
Abstract
Out-of-distribution (OOD) detection is critical for safe deployment of medical AI systems. Recently, test-time adaptation (TTA) has emerged as a new paradigm for OOD detection, automatically adjusting detector behavior during deployment. However, such automatic adaptation mechanisms may raise safety concerns in safety-critical clinical environments. While physician oversight can mitigate these risks, it is resource-intensive and must be judiciously allocated. To reconcile safety with efficiency, we propose CHILD, a training-free framework designed to enhance streaming OOD detection via sparse human feedback. Operating under strict budget constraints, CHILD employs an adaptive risk-aware sample selection mechanism to pinpoint only the most decision-uncertain samples for review. Crucially, it maximizes the utility of this sparse feedback through a retrieval-based score calibration module, which refines model predictions using a compact feature cache without any parameter updates. Extensive experiments on four medical benchmarks demonstrate that CHILD turns limited supervision into significant reliability gains: with a sparse feedback budget of only 5%, it reduces the average FPR95 from 72.63% to 60.26% and improves AUROC from 75.53% to 81.85%, consistently outperforming state-of-the-art baselines. Our code is publicly available at https://github.com/figec/CHILD.