DINOv3 model improves liver fibrosis staging using medical image patches
DINO-Med: A Unified Patch-Based Adaptation Framework for Multi-Modal Medical Image Analysis Applied to Liver Fibrosis Staging
Computer Vision and Pattern Recognition
Summary
It is hard to use AI models made for everyday photos on medical images because medical scans look very different. The authors created a system that breaks medical images into small pieces and analyzes them to detect liver fibrosis stages. They compared different ways of representing image parts and found that a model called DINOv3 worked best. Their system was tested on 360 patients and showed more accurate detection of mild fibrosis and cirrhosis than other methods.
What this means in practice
- •For medical imaging teams: Enable more accurate liver fibrosis staging from multimodal medical scans by integrating patch-based DINOv3 feature analysis.
- •For healthcare ai developers: Develop AI tools for liver disease assessment that handle multimodal scans without retraining large models by using the proposed patch-based framework.
Authors
Boya Wang, Ruizhe Li, Chao Chen, Xin Chen
Abstract
Adapting natural-image foundation models like DINOv3 to multi-modal medical imaging is challenging due to the significant domain gap between natural color images and multi-channel medical scans. We present a unified, patch-based framework that processes raw multimodal imaging through training-free registration, automated localization, and mask-filtered patch extraction. This architecture culminates in a hierarchical strategy that aggregates patch-level insights into subject-level diagnostics. Using liver fibrosis staging as a case study, we evaluate four patch-level feature representations: handcrafted Radiomics features, learned ResNet features, pre-trained foundation model SAM-Med2D features, and frozen DINOv3 features. To ensure a controlled comparison, all models utilize the same lightweight MLP head and are evaluated across both rigid and deformable registration settings. Our training protocol focuses on mild fibrosis (S1) and cirrhosis (S4) classes only, enabling a single classifier to address both substantial fibrosis detection and cirrhosis staging. Evaluated via 10 random train (90%)/ test (10%) splits on 360 subjects from the CARE 2025 Liver Track 4 cohort, our DINOv3-based framework significantly outperforms all baselines, achieving the best classification accuracy of 78.4% for S1 and 75.8% for S4.