SAMRI-3D: Adapting SAM2 for 3D MRI Segmentation with Global Volume Tokens
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created SAMRI-3D, a large test and improved method for segmenting 3D MRI scans using a model called SAM2. They showed that by keeping the image understanding part fixed and fine-tuning a smaller part, their model worked much better than previous similar models. To help with tricky areas that are hard to see on single slices, they introduced a technique called Global Volume Tokens trained with a special objective, improving accuracy without slowing down the model. Their approach performed well across many datasets, including ones not seen during training. They plan to share their benchmark, code, and models with others.
Foundation modelsSegment Anything Model 2 (SAM2)MRI segmentation3D medical imagingDice scoreGlobal Volume Tokens (GVT)Truncated Signed Distance Field (TSDF)Fine-tuningZero-shot learningBenchmark dataset
Authors
Zhao Wang, Wei Dai, Hongfu Sun, Craig Engstrom, Shekhar S. Chandra
Abstract
Foundation models such as Segment Anything Model 2 (SAM2) have transformed natural-image and video segmentation, and recent work has begun adapting them to medical imaging. These adaptations, however, are largely general-purpose models that treat MRI as one modality among many; large-scale, MRI-specific modelling and benchmarking remain limited, even though MRI's low soft-tissue contrast leaves many boundaries effectively invisible on individual slices. We present SAMRI-3D, a benchmark and method for 3D MRI segmentation with SAM2. The SAMRI-3D benchmark is the largest MRI-only evaluation to date - 10,392 volumes from 34 datasets (27 public, 7 in-house) spanning 12 anatomical domains and 10+ sequences, with explicit seen/unseen splits. Freezing the image encoder and fine-tuning only the lightweight decoder and memory modules raises mean Dice from 0.58 (zero-shot SAM2) to 0.76, surpassing recent SAM-based medical models (SAMed-2 0.69, Medical-SAM2 0.49, SAM-Med3D 0.37) with strong statistical significance. To target invisible boundaries, we introduce Global Volume Tokens (GVT): persistent memory tokens trained with a Truncated Signed Distance Field (TSDF) reconstruction objective that is discarded at inference (zero added cost). This full model, SAMRI-3D, attains the best accuracy (0.78) and lowest variance across all 34 datasets and, uniquely, shows no drop on 8 held-out datasets (0.79 unseen vs. 0.78 seen); per-sequence analysis confirms the TSDF objective helps most where per-slice contrast is weakest. We will release the benchmark, code, and models in this paper.