Cardiac MRI model learns where to look for better diagnosis
Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models
Computer Vision and Pattern Recognition
Summary
Interpreting heart MRI images is hard because doctors must focus on specific heart parts tied to the medical question. The authors created a large set of heart images linked to anatomy and clinical questions to train computers to recognize heart structures and answer questions without needing detailed reports for each image. They introduced a new method called CARA that helps the model focus on the right heart regions based on the question being asked. Their combined approach improves how well the model understands heart MRI images and works well even on data from different hospitals.
What this means in practice
- •For medical imaging teams: Create AI tools that can accurately identify heart structures and answer clinical questions from MRI scans without needing detailed reports for each scan.
- •For radiology software developers: Build diagnostic software that uses anatomically guided attention to improve heart region localization in cardiac MRI interpretation workflows.$Commercial implications: Enables software vendors to offer enhanced AI-assisted cardiac MRI tools with better accuracy and interpretability for clinical use.
Authors
Bangwei Guo, Xiao Chen, Boris Mailhe, Jia Yao, Yiqing Wang, Ankush Mukherjee, Yikang Liu, Zheyuan Zhang, Hang Yu, Terrence Chen, Shanhui Sun
Abstract
Cardiac magnetic resonance imaging (CMR) enables assessment of cardiac anatomy, ventricular function, and myocardial tissue characteristics. Clinicians interpret these images by identifying cardiac structures and focusing on the regions relevant to each clinical question, motivating anatomically guided vision-language models (VLMs). Yet CMR-specific supervision for anatomical localisation and clinical question answering remains limited. To address this gap, we investigate fine-grained CMR visual question answering through anatomical grounding and guided attention. We construct 128,915 anatomical-grounding and 42,799 clinical QA pairs across short-axis cine, late gadolinium enhancement, and long-axis cine. These datasets support anatomical recognition, localisation, and clinical assessment without requiring paired reports for individual training images. To help the model learn where to look, we introduce Cardiac Anatomy-Routed Attention (CARA), which selects predicted anatomical priors according to the question and guides decoder attention with learned task-specific strengths. Combining anatomical grounding pretraining with CARA yields our model, CARA-VL. Experiments demonstrate CARA-VL's strengths in clinical assessment and regional localisation across CMR imaging settings, with promising generalization to an external clinical cohort. Together, our data and method provide a practical framework for studying and advancing cardiac visual understanding in VLMs. We will release the QA data derived from public datasets upon publication.