Frozen model ensembling improves medical image segmentation across vendors
When Online Adaptation Hurts: Parameter-Frozen Test-Time Ensembling for Continual Medical Image Segmentation
Computer Vision and Pattern Recognition
Summary
Medical image models often perform worse when used on data from different hospitals or machines than those they were trained on. The authors show that instead of changing the model during use, simply making multiple predictions on transformed versions of the same image and averaging them can improve accuracy. This method, called parameter-frozen inference enhancement, works better than other adaptation techniques and keeps important details clearer. Their findings caution that adapting models on changing data streams can introduce errors compared to a well-designed fixed approach.
What this means in practice
- •For medical imaging teams: Improve segmentation consistency on medical scans from different machine vendors without retraining models during deployment.
- •For radiology software developers: Build segmentation tools that maintain performance across diverse clinical settings by using frozen ensembled inference instead of continual model updates.
Authors
Ruijie Huang
Abstract
Medical image segmenters often get worse when sites, scanner vendors, or protocols change. Continual test-time adaptation (CTTA) addresses this problem without target labels, but it can be impossible to update a model on a non-stationary stream and can lead to a lot of errors. We examine a more reasonable and meaningful alternative: parameter-frozen inference enhancement(PIE). We use a source-trained segmenter that learns about anatomy-preserving scale and flip views, maps their predictions back to the native location, and averages the probabilities. We do not modify the weights of the model or the normalization statistics. On a cardiac MRI stream from M\&Ms, which is trained on vendor A and evaluated sequentially on vendors B, C, and D, PIE has 0.7786 mean Dice, compared to 0.7680 for source-only inference and 0.7388--0.7416 for five other online-adaptation baselines. The controlled ablations show that performance saturates at 28 views, and confidence weighting, class-prior correction, connected-component filtering, morphological refinement, and inter-slice smoothing have no effect or cause negative transfer. Qualitative results on cardiac MRI and fundus images are also consistent with the frozen ensemble keeping thinner and nested anatomical structures. These results provide a strong, stable baseline for medical CTTA and expose an important failure mode: adaptation and handcrafted refinement can be less reliable than carefully designed inference.