Text guided system improves medical image segmentation accuracy
Text-Guided Medical Image Segmentation via Residual Aggregation and Dense Alignment Representations
Computer Vision and Pattern Recognition
Summary
Medical images are hard to analyze when lesions are hard to see or boundaries are unclear. The researchers developed a method called RADAR that helps computers understand text instructions better to mark areas in medical images more accurately. It combines different specialized tools and adapts quickly without needing many new settings. Tests show it works well on various medical image datasets while needing fewer parameters to learn.
What this means in practice
- •For medical image software developers: Create segmentation tools that interpret doctors' textual instructions to outline regions in complex medical scans more reliably.
- •For healthcare ai teams: Integrate parameter-efficient models for adapting text-based segmentation methods to diverse medical imaging tasks without extensive retraining.
- •For automated diagnostic device makers: Develop devices with improved lesion detection by leveraging text-guided image segmentation for ambiguous or low-contrast conditions.$Commercial implications: Enables commercial AI-powered diagnostic products that better detect and mark hard-to-see lesions using text instructions.
Authors
Jie Gui, Hang Tu, Yunfeng Dou, Wen Sha, Xiuquan Du, Shuo Li
Abstract
Specifying segmentation targets with natural language provides flexible semantic conditioning for medical image segmentation. However, existing methods still struggle to reliably ground high-level textual semantics into accurate pixel-level spatial prompts, especially under low-contrast lesions, ambiguous boundaries, and visually similar anatomical contexts. In this paper, we propose RADAR (Residual Aggregation and Dense Alignment Representations), a parameter-efficient framework for text-guided medical image segmentation. RADAR explicitly decouples and synergizes the medical semantic anchoring capability of BiomedCLIP with the prompt-driven mask decoding ability of MedSAM2, while enabling low-cost task adaptation via LoRA. Specifically, Guided Residual Depth Aggregation adaptively aggregates cross-layer features under semantic guidance to alleviate delayed textual injection and deep semantic dilution. Structured Dense Representation Alignment leverages a frozen BiomedCLIP to construct crop-level target, boundary, and context anchors, improving the local discriminability of dense features and the reliability of automatic prompts. Multi-modal Prompt Generation further converts cross-modal semantic responses into composite text, point, and mask prompts for MedSAM2 decoding. Experiments on three public medical image segmentation datasets show that RADAR achieves competitive performance with only 0.74M trainable parameters, demonstrating its effectiveness in parameter-efficient adaptation, semantic grounding, and dense representation enhancement. The source code is available at https://github.com/JGuiidev/RADAR.