Multi modal medical image segmentation improves with corruption aware fusion
When Fusion Fails: Corruption-Aware Rebalanced Fusion for Multi-Modal Medical Image Segmentation
Computer Vision and Pattern Recognition
Summary
Multi-modal medical imaging combines different types of scans to help doctors see more details, but sometimes using all images together works worse than using just one when some scans are lower quality. The authors found that poor-quality scans can confuse the system during training and prediction due to how images are processed and combined. They propose a new method called CoReFuse-Med that reduces the impact of corrupted features and balances the importance of each scan type during analysis. Their tests show this method works better and is more reliable when some images have lower quality.
What this means in practice
- •For medical imaging engineers: Improve multi-modal segmentation models to maintain accuracy when some input scans have lower resolution or noise.
- •For automated radiology software developers: Develop more reliable software that better handles inconsistent scan quality across different imaging types for clinical use.$Commercial implications: Enables building robust multi-modal medical imaging tools that can be marketed to hospitals for improved diagnostic support.
Authors
Yuchen Pei, Xiaoyu Hu, Yixiong Zou, Dingwen Hu, Hui Chu, Yutao Ma, Shijun Qiu, Gang Li
Abstract
Multi-modal medical image segmentation leverages complementary diagnostic information, yet fusion can underperform single-modality baselines when spatially aligned inputs differ in quality. Here, "corruption" primarily denotes resolution-induced degradation rather than misalignment or complete modality absence, while synthetic noise is evaluated only as an auxiliary setting. We identify a critical optimization-inference inconsistency: degraded modalities can receive weak training updates yet substantially affect predictions, indicating active interference with fusion. We attribute this failure to resampling-induced feature corruption and optimization bias, where noisy features propagate through skip connections and encourage unreliable modality selection. We therefore propose CoReFuse-Med, a Corruption-aware Rebalanced Fusion framework that suppresses corruption during feature transmission and rebalances modality contributions during high-level fusion. Experiments on EPVS, BraTS, and WMH, including multiple Z-axis slice-retention ratios and an auxiliary noise test, demonstrate improved accuracy and robustness under modality-quality discrepancies. Our code is available at https://github.com/lrever/CoReFuse.