Brain2Vec improves brain MRI analysis with large scale pretraining
Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Analyzing brain MRI scans with computers can be hard because labeled examples for training are limited. The authors created Brain2Vec, a computer model trained on over 74,000 brain scans, to learn general brain features without needing labels. This model helps speed up and improve tasks like estimating a person's brain age or identifying brain conditions. Brain2Vec also reveals meaningful brain regions related to these features and can be used easily through open-source code or an online platform.
What this means in practice
- •For medical imaging teams: Improve brain MRI diagnostic models by initializing with Brain2Vec to accelerate training and enhance accuracy on tasks like brain age and disease classification.
- •For cloud platform operators: Offer brain MRI feature extraction services via cloud platforms using Brain2Vec, avoiding the need for local installation or specialized hardware.$Commercial implications: Enables scalable cloud-based brain MRI analysis services that can be sold to hospitals and imaging centers.
Authors
Spiros Maggioros, Guray Erus, Gareth Harman, George Aidinis, Alexander Getka, Aristeidis Sotiras, Christos Davatzikos
Abstract
Deep learning models have become essential tools in medical imaging, yet training them end-to-end for diverse tasks remains challenging due to limited labeled data and computational constraints. To address this, we introduce Brain2Vec, a foundation model pre-trained on 74,425 T1-weighted Magnetic Resonance Imaging (MRI) scans pooled from multiple studies. Brain2Vec can serve both as a weight initialization backbone for fine-tuning downstream tasks and as a standalone feature extractor. We evaluated Brain2Vec in three tasks: downstream predictive performance, convergence speed, and embedding quality. Using Brain2Vec's pretrained weights as initialization and fine-tuning on labeled data for a brain age prediction task, results demonstrate improved final accuracy and faster convergence compared to random initialization. Additionally, its frozen embeddings outperform anatomically defined volumetric features on four of five downstream classification tasks and perform competitively with a state-of-the-art foundation model pre-trained on substantially more data. Lastly, we show that Brain2Vec encodes information primarily from a spatially localized and neuroanatomically meaningful set of brain regions, and that manipulating a subset of latent features produces monotonic, region-specific effects on the reconstructed anatomy. Brain2Vec code and trained models are available as an open-source package on https://github.com/CBICA/Brain2Vec. Users can apply Brain2Vec locally by installing the package, or directly through the NiChart cloud platform https://cloud.neuroimagingchart.com, where Brain2Vec features can be derived without requiring local installation or specialized infrastructure.