Imputing cancer molecular signatures from standard pathology images
Cross-Modal Contrastive Learning for the Retrieval of Immunotherapy-Associated Molecular Signatures from Histopathology
Machine LearningArtificial IntelligenceInformation Retrieval
Summary
Gastric adenocarcinoma is a deadly cancer type where response to immunotherapy can be predicted by certain gene activity patterns, but detecting these requires expensive tests. The authors developed a method called CCMIL that can find these molecular signatures just by looking at regular stained tissue images. It works by matching visual patterns with gene activity, allowing doctors to search similar cases and guess molecular traits without costly RNA testing. This method also helps highlight which parts of the tissue relate to inflammation, supporting faster and cheaper clinical assessments.
What this means in practice
- •For hospital pathology labs: Provide molecular subtype pre-screening for gastric cancer patients from standard pathology slides without needing expensive genetic tests.
- •For digital pathology software developers: Build search tools that retrieve similar histopathology cases with matching molecular characteristics to support pathologists' diagnoses.
Authors
Sigrid Vila-Bagaria, Mar Teixidó, Miquel Piñol, Felip Vilardell, Robert Montal, Veronica Vilaplana
Abstract
Gastric Adenocarcinoma is a leading cause of cancer mortality. Although "Inflamed/Non-Inflamed" subtypes have been proposed to predict immunotherapy response, their identification relies on a costly 10-gene RNA signature. We propose a Cross-modal Contrastive Multiple Instance Learning (CCMIL) framework for cross-modal retrieval, imputing these molecular signatures directly from standard Hematoxylin & Eosin (H&E) slides. By leveraging a supervised contrastive objective, CCMIL aligns visual morphological patterns with molecular phenotypes into a shared latent space. This establishes an interpretable search-by-case retrieval engine, enabling pathologists to query a whole slide image to surface transcriptomically coherent neighbors and approximate RNA signatures without genomic sequencing at inference. Our results demonstrate that this retrieval-first approach captures the continuous phenotypic spectrum of tumor inflammation and yields clinically interpretable attention heatmaps. Furthermore, the learned representation also supports competitive downstream classification, providing a practical molecular pre-screening strategy.