Knee3DVLM improves knee MRI diagnosis using dual-sequence imaging
Knee3DVLM: Dual-Sequence Full-Volume Vision-Language Modeling for Comprehensive Knee MRI Assessment
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Doctors use MRI scans to check knees for problems like arthritis, but these scans come in different types that show different details. The authors created a new computer model called Knee3DVLM that looks at two types of knee MRI scans together, improving its ability to spot 57 different knee issues. This model was tested on over a thousand knee exams and outperformed versions that used just one type of scan. This approach could help provide more accurate reports about knee health.
What this means in practice
- •For medical imaging developers: Develop software that analyzes dual-sequence full-volume knee MRIs to generate detailed diagnostic reports automatically.
- •For radiology departments: Integrate dual-sequence vision-language MRI models to assist radiologists in identifying and reporting on multiple knee pathologies.
Authors
Maryam Baizhigitova, Andrew Seohwan Yu, Po-Hao Chen, Naveen Subhas, Sixu Chen, Xinxin Wang, Kunio Nakamura, Richard Lartey, Xiaojuan Li, Mingrui Yang
Abstract
Vision-language models (VLMs) are increasingly being applied to three-dimensional medical imaging, but their application to knee MRI remains limited, particularly for interpreting the complementary sequences used in clinical practice. We introduce Knee3DVLM, a sequence-aware VLM that uses full-volume DESS and fluid-sensitive TSE MRI to predict 57 anatomically resolved binary diagnostic targets derived from the MRI Osteoarthritis Knee Score (MOAKS) for structured reporting. We evaluated DESS-only, TSE-only, and paired DESS-TSE configurations using subject-disjoint Osteoarthritis Initiative partitions. In a held-out cohort of 1,074 examinations, the fused model achieved 72.98% average accuracy, 71.17% balanced accuracy, 78.96% mean ROC-AUC, and 78.74% macro ROC-AUC, the highest values among the three configurations. In a secondary multiclass analysis aligned with the released 3DReasonKnee cohort, Knee3DVLM was numerically higher than the strongest reported 3DReasonKnee configuration across five pathology categories. These findings support dual-sequence full-volume modeling for comprehensive knee MRI assessment.