Lesion aware attention improves mammogram segmentation with mixed label types
Lesion-Aware Attention for Mammography Segmentation with Heterogeneous Supervision
Computer Vision and Pattern Recognition
Summary
Identifying and outlining breast lesions in mammograms is important for detecting cancer, but detailed labels showing exact lesion areas are hard and expensive to get. The authors address this by using both detailed lesion outlines (when available) and overall image labels that say if a lesion type is present. They designed a system with two learning parts: one that learns from detailed masks and another that learns from image-level labels, using a special attention method to focus on lesion areas and ignore background. Their method improves segmentation of both common and rare lesion types, making it easier to spot small or subtle findings in breast images.
What this means in practice
- •For medical imaging software developers: Create mammogram analysis tools that segment a wider range of lesion types using limited detailed annotations and more abundant image-level labels.$Commercial implications: Enables development of enhanced breast cancer diagnostic tools that reduce reliance on costly pixel-level annotation and improve lesion detection.
- •For radiology department it teams: Improve integration of mammography CAD systems to better identify subtle and rare lesions by leveraging mixed types of annotation available in hospital imaging archives.
Authors
Di Dai, Bo Liu, Liwei Wang
Abstract
Accurate lesion segmentation in mammography is important for computer-aided breast cancer diagnosis, but dense pixel-level annotations remain costly and limited. Existing public segmentation datasets usually cover only a small set of lesion types with manually delineated masks, whereas larger mammography datasets often provide only image-level diagnostic labels. This annotation mismatch makes it difficult to train segmentation models that can generalize to rare or weakly annotated lesion categories. To address this problem, we study open-vocabulary mammographic lesion segmentation under a mixed-supervision setting, where pixel-level masks are available for base lesion categories and image-level labels are available for a broader set of categories. We propose a two-branch framework that jointly learns from segmentation data and classification data. The segmentation branch learns lesion boundaries from annotated masks, while the classification branch introduces additional image-level semantic supervision. To better transfer image-level supervision to lesion-relevant regions, we further introduce a Lesion-wise Attentive Classifier that maintains category-specific feature prototypes and generates lesion-aware attention maps for foreground feature pooling. This design reduces the influence of background tissue and encourages more compact lesion representations. Experiments on multiple mammography datasets show that the proposed method consistently improves open-vocabulary segmentation baselines on both base and novel lesion categories, while reducing the gap to fully supervised models that require dense target-domain masks. Qualitative results further indicate that our method produces more complete lesion localization for small, low-contrast, and visually subtle mammographic findings.