Heart disease segmentation methods perform differently on new patient data
Beyond In-Distribution Metrics: A Systematic Out-of-Distribution Evaluation of Congenital Heart Disease Segmentation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Segmenting images of hearts for congenital heart disease helps doctors plan surgeries but usually requires lots of manual work. The authors found that popular computer programs perform well on familiar data but struggle on new data from different hospitals or scanners. One newer method generalized better to new patient images, and giving it a few new examples from the target hospital improved results a lot. This study suggests evaluating medical image methods on new data, not just on familiar images, to better understand how well they work in real clinics.
What this means in practice
- •For medical imaging teams: Improve robustness of heart segmentation tools by testing algorithms on data from new hospitals and scanners before clinical use.
- •For healthcare software developers: Develop segmentation systems that generalize better to diverse clinical data by integrating architectures like SwinUNETR with small labeled target domain data.$Commercial implications: Enables creation of more reliable congenital heart disease imaging software that adapts efficiently to different hospitals' imaging protocols.
Authors
Aniketh Vijesh, Shrisharanyan Vasu, Abhijit Ramesh, Clare Pomeroy-Ward, Harikrishnan Anil Maya, Sarin Xavier, Mahesh Kappanayil, Gilad Gressel
Abstract
Congenital heart disease (CHD) diagnosis and surgical planning often require patient-specific 3D anatomical models, but manual segmentation is labor-intensive, particularly in complex anatomies. Although deep-learning methods can automate this process, they are typically evaluated in-distribution, despite clinically relevant shifts in scanner, protocol, institution, population, and imaging modality. We present, to our knowledge, the first systematic evaluation of out-of-distribution (OOD) generalization in CHD segmentation, using ImageCHD as a held-out target cohort. We compare representative segmentation architectures under combined CT and CMR training, CT-only training, self-supervised pretraining, and limited target-domain adaptation. In-distribution performance proves to be a poor indicator of cross-cohort robustness: nnU-Net achieves the highest validation Dice (0.77) but falls to 0.51 on ImageCHD, while SwinUNETR generalizes substantially better, reaching 0.67 Dice. MAE and JEPA pretraining provide only modest additional benefit, suggesting that architecture contributes more to robustness than the tested pretraining strategies in this setting. When limited target-domain supervision is introduced, all SwinUNETR variants exceed 0.76 Dice with only 11 labeled ImageCHD cases. These findings demonstrate that conventional in-distribution evaluation can obscure clinically important generalization failures and support explicit cross-dataset testing as a key component of CHD segmentation evaluation.