Unified AI segments pancreas in CT and MRI with fewer labels
Unified CT and MRI Pancreas Segmentation for Label-Efficient Cross-Modality Subregion Transfer
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Segmenting the pancreas in medical images is hard because CT and MRI scans look very different. The authors created a single AI model that learns to work well on both types of scans by training on a large mix of them. This model then uses what it learned from both CT and MRI data to identify parts of the pancreas in MRI scans, even when only a few examples of these parts are labeled. Their method works well on different datasets and helps reduce the amount of labeled MRI data needed for detailed pancreas segmentation.
What this means in practice
- •For medical imaging developers: Create pancreas segmentation tools that work reliably on both CT and MRI scans using fewer labeled MRI images for subregion analysis.
- •For clinical workflow engineers: Develop imaging software to assist radiologists with consistent pancreas segmentation across CT and MRI modalities in diagnostic workflows.
Authors
Ziliang Hong, Hongyi Pan, Halil Ertugrul Aktas, Andrea Bejar, Elif Keles, Frank H. Miller, Michael B. Wallace, Rajesh N. Keswani, Gorkem Durak, Ulas Bagci
Abstract
Robust medical image segmentation across imaging modalities is challenging because of large differences in appearance and intensity distributions. Models trained on a single modality often show substantial performance drops when applied to unseen domains. In this work, we develop a unified 3D pancreas segmentation framework that applies domain-adversarial learning to 4,604 heterogeneous CT and MRI scans to learn anatomical representations. A shared nnU-Net encoder-decoder is trained for whole-pancreas segmentation, with a latent domain discriminator encouraging CT-MRI feature alignment. The learned encoder is subsequently transferred to pancreatic head-body-tail segmentation using limited MRI-only subregion annotations. An average Dice score of 87.31% on the in-distribution test set and Dice scores ranging from 84.20% to 88.09% across external OOD datasets were achieved in whole pancreas segmentation. Dice scores of 80.53% on MRI and 83.05% on CT were achieved for downstream subregion segmentation, without using CT subregion annotations. These results demonstrate that a unified anatomical representation can support both cross-modality pancreas segmentation and label-efficient downstream transfer.