Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification
Computer Vision and Pattern Recognition
Summary
The authors developed a new method to help computers identify and classify breast cancer lesions in mammogram images by focusing on specific regions rather than the whole image. Their approach mimics how doctors look closely at small, important areas described by medical text to improve accuracy. They trained the system to match image regions with clinical descriptions and added special techniques to avoid confusion from irrelevant parts of the image. Tests on two different mammogram datasets showed their method works better than previous ones across various settings.
Authors
Zhengbo Zhou, Jiren Li, Dooman Arefan, Margarita Zuley, Shandong Wu
Abstract
Vision-language models trained with contrastive objectives have shown promise in medical image analysis. However, conventional global image-text alignment is ill-suited for mammography, where diagnostically relevant lesions are spatially localized and occupy only a small fraction of the image. Subtle morphological cues critical for malignancy assessment can be diluted when representations are learned at the whole-image level. In this work, we propose a novel region-grounded vision-language learning method for detection-guided mammographic lesion classification. The method mirrors radiologists' diagnostic paradigm. First, a region-text contrastive pretraining stage aligns lesion-specific features with structured clinical descriptors derived from radiology metadata. To mitigate semantic collapse and background bias in low-vocabulary settings, we introduce a multi-component objective incorporating positive alignment, fine-grained semantic hard negatives, and background suppression. Second, an auxiliary lesion detection head is jointly optimized with contrastive classification to preserve spatial sensitivity and enable localization-aware malignancy classification. Extensive experiments on two independent datasets, CBIS-DDSM and VinDr-Mammo, show superior performance of our method compared to related methods under in-domain, cross-dataset, and transfer learning settings.