Pet specific dinov2 model improves imaging analysis and prognosis
DINOv2 for PET: A Pre-Training Domain Benchmark Across Classification, Segmentation, and Prognosis
Computer Vision and Pattern Recognition
Summary
Medical images from PET scans are harder to analyze with general image tools than other scans like MRI or CT. The authors created a version of DINOv2, a method that teaches computers to understand images without labels, specifically trained on PET images. Their PET-DINOv2-2D model performs as well or better than previous models trained on natural or other medical images across tasks like recognizing abnormal scans, outlining lesions, and predicting patient outcomes. This new model works especially well when there are fewer labeled examples, making it efficient in real clinical settings.
What this means in practice
- •For medical imaging teams: Improve PET scan analysis accuracy and efficiency in tasks like lesion detection and outcome prediction using a model pre-trained specifically on PET.
- •For clinical ai developers: Build AI tools that perform better with fewer labeled PET images by using the PET-DINOv2-2D model for pre-training.
Authors
Roxanne De Maeyer, Yannick Brackenier, Vicky De Ridder, Simon DeKeyser, Maarten Larmuseau, Jef Vandemeulebroucke
Abstract
DINO (self-DIstillation with NO labels) has become a stan- dard for self-supervised pre-training, producing features that generalise across datasets and downstream tasks. In medical imaging it has been adopted mainly for radiology (X-ray, CT and MRI), where even natural- image DINO features transfer well. In the literature, these features have been reported to transfer worse to functional imaging such as PET. Yet PET-specific DINO models have barely been introduced, and even less validated across multiple tasks and datasets. We present PET-DINOv2-2D, the first DINOv2 model pre-trained on 2D PET, and benchmark it against open-source encoders: the origi- nal DINOv2 (pre-trained on the LVD-142M natural-image dataset) and OmniRad (pre-trained on CT, MRI and ultrasound). Using the same lightweight heads for every backbone, and despite far less pre-training data, PET-DINOv2-2D matches or outperforms both on three down- stream PET tasks: normal/abnormal classification, lesion segmentation and recurrence-free survival prognosis. Its advantage is largest when few labels are available, suggesting that PET pre-training gives more transferable, label-efficient features. Code is made available at https://github.com/nuclivision/pet-dinov2-2d/.