MRI model improves anatomy mapping and abnormality detection accuracy
A4-MR: Abnormality-aware Anatomical Alignment for MR
Computer Vision and Pattern Recognition
Summary
MRI scans can be hard to understand because the brain or body parts look different in each picture. The authors created a new computer model called A4-MR that learns to identify and match parts of the body in 3D MRI images and also knows when something looks abnormal. They trained it with many MRI scans and doctor reports to help it recognize both healthy and unhealthy body parts. This model can do better than previous ones at finding problems and outlining organs in MRI pictures.
What this means in practice
- •For medical imaging teams: Use the model to improve automated detection of abnormalities and segmentation of organs in MRI scans with better accuracy than previous methods.
- •For radiology software developers: Integrate the model to enhance MRI analysis tools with anatomy-aware image and text alignment to support localization and pathology-related tasks.$Commercial implications: Allows selling advanced MRI diagnostic software that improves clinical workflow by combining image understanding with radiology report data.
Authors
Noel D'souza, Zhijian Yang, Nishank Singla, Pranay Doshi, Krisztian Koos, Fei Tan, Bruno Nunes, Laszlo Rusko, Bernadett Kolozsvari, Eszter Sabo, Emanuele Valeriano, Vikash Gupta, Seyed Iman Zare Estakhraji, Marc Lebel, Farzin Haddadpour, Taha Kass-Hout, Erhan Bas
Abstract
Recent advances in large-scale foundation models have enabled strong generalization across a wide range of clinical and research tasks in magnetic resonance imaging (MRI). However, existing vision-language MRI models remain limited in anatomy-specific localization and pathology-aware representation, particularly in real-world settings. We introduce A4-MR, a 3D vision-language foundation model trained on approximately 50,000 MRI studies comprising 491,000 series, roughly half of which are paired with radiology reports. A4-MR is designed to: (1) learn anatomy localization through large-scale pretraining with noisy labels; (2) align image and text representations at the anatomical level; and (3) learn distinct representations of healthy and abnormal anatomy. Across a diverse set of downstream tasks, A4-MR outperforms existing foundation models, including a +1.7 AUROC improvement over MedSigLIP for abnormality detection. Using a frozen encoder, A4-MR also surpasses an end-to-end task-specific MedNeXt model for knee segmentation by +1.4 Dice and performs comparably to the best task-specific model for abdominal organ segmentation on AMOS, within 0.4 Dice. Finally, without anatomical prompts, A4-MR localizes 3D anatomy comparably to MedSAM2 provided with ground-truth bounding boxes. Together, these results demonstrate that anatomy-level multimodal alignment and abnormality-aware representation learning can enable a single MRI foundation model to support localization, segmentation, and pathology-related tasks.