Foundation model embeddings detect early changes in mammograms before cancer diagnosis
Foundation model embeddings capture pre-diagnostic changes on screening mammograms
Computer Vision and Pattern RecognitionMachine Learning
Summary
Changes in breast tissue can show up on mammograms before cancer is officially diagnosed. The paper shows that certain AI models, trained on medical images, can capture these early changes without being specifically taught to find cancer. The researchers compared different AI models and found some that can track these subtle changes better than others. This finding could help identify cancer earlier through routine mammogram screenings.
What this means in practice
- •For radiology teams: Detect early tissue changes in mammograms to improve cancer screening accuracy before diagnosis.
- •For medical ai developers: Create AI tools that use foundation model embeddings to flag subtle cancer-related changes without needing specialized training data.$Commercial implications: Enables development of pre-diagnostic cancer detection tools for healthcare providers, offering improved screening support.
Authors
Kalina P. Slavkova, Eric Brattain, Aditya Gowd, Akash Pattnaik, Jean-Benoit Delbrouck, Matthew Morgan, Julie Bauml, Javid Abderezaei, Khan Siddiqui
Abstract
Foundation model embeddings of screening mammograms may encode pre-diagnostic tissue change without task-specific adaptation. We tested whether embeddings move faster along a data-derived "cancer direction" in women later biopsied for cancer than in matched screen-negative controls, and whether this depends on pretraining domain. We studied 1,773 biopsied women (785 malignant, 988 biopsy-negative) and 1,773 matched controls, each with at least two annual screening exams before their index exam. An identical pipeline was applied to four 2D models: Mammo-CLIP (MC, out-of-distribution mammography), HOPPR (in-distribution mammography), MedImageInsight (MII, general medical imaging), and BiomedCLIP (biomedical vision-language pretraining on literature figures). Breast-level embeddings quantified longitudinal movement along the cancer direction. We compared cases and controls using a between-patient design with complementary mixed-effects analysis, and biopsied versus healthy contralateral breasts within patients. Under matched modality in MII embedding space, malignant cases drifted significantly faster than controls in the first two screening intervals preceding the index exam; biopsy-negative cases showed significance only in the first. MC differences were significant in the first interval for both biopsy groups. Within-patient comparisons showed a broadly similar pattern, with MC significance extending to the second interval in both groups and HOPPR showing significance at interval 1. BiomedCLIP showed no significant differences in either design or biopsy group. Overall, directional embedding velocity emerges as a property of clinically grounded rather than general biomedical pretraining, showing that foundation model embeddings can encode pre-diagnostic mammographic change without task-specific adaptation.