Vessel detection improves with reference graph embeddings for 2d medical images
ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting blood vessels in medical images is important but difficult because vessels can be very thin and look different across scans. The authors improved a popular segmentation model called SAM by adding special features from reference graphs that represent vessel shapes and patterns. These features help the model better recognize vessels, especially tiny ones, across many types of images. Tests on lots of datasets show that their method works better than existing ones.
What this means in practice
- •For medical imaging software developers: Build vessel segmentation tools that achieve higher accuracy on diverse 2D scans using reference graph-driven embeddings.
- •For clinical imaging teams: Improve vessel visualization and analysis workflows by integrating models that better detect thin and complex vascular structures.
Authors
Donghang Lyu, Zichen Zhang, Oleh Dzyubachyk, Marius Staring
Abstract
Vessel segmentation in medical images is essential for many clinical tasks, ranging from diagnosis to treatment planning. However, it remains challenging due to complex vascular morphology and diverse imaging conditions. Existing deep learning methods rarely aim at building a generalizable vessel segmentor across anatomies and modalities. While the Seg- ment Anything Model (SAM) has shown promise for med- ical image segmentation, its original design does not fully exploit vascular morphology and struggles with fine-grained vascular structures, leading to suboptimal performance. In this paper, we propose ReG-SAM, a SAM-based framework tailored to 2D vessel segmentation that leverages reference graph set for enhancing vascular representations. Specifically, we introduce two modality-aware representations derived from the reference masks: graph prompt embeddings (GPEs) that encode global spatial features from graphs, and vascu- lar prototype embeddings (VPEs) that capture fine-grained modality-specific vessel characteristics from multi-scale fea- ture maps and vascular masks. Since both require vascular masks that are unavailable during inference and require robust modality-aware vascular feature representations, we construct a modality-wise vascular database and develop two reference graph-guided representation learning schemes for estimating GPEs and VPEs using samples from the database rather than ground-truth masks. Extensive experiments across 19 datasets demonstrate that ReG-SAM consistently outperforms existing baselines, even those using manual prompts, particularly on challenging thin vessels.