Pathology models predict gene activity patterns from tissue images

Predicting Single-Cell Spatial Gene Expression from H&E using Pathology Foundation Models

Computer Vision and Pattern Recognition

Summary

Predicting gene activity in individual cells usually requires costly and complex tests. This paper shows that computer models can estimate gene expression from standard microscope images of tissue stained with common dyes. The authors improved existing models so they better capture detailed cell features, achieving stronger correlation with actual gene data. Their method works best for genes that vary a lot in space, revealing how tissue appearance relates to molecular activity.

What this means in practice

  • For pathology labs: Estimate spatial gene expression profiles directly from routine H&E images to add molecular insights without extra costly tests.
  • For biotech companies: Develop diagnostic tools integrating morphology-based gene expression predictions to support precision medicine workflows.$Commercial implications: Enables software products that infer molecular information from standard pathology images for clinical and research customer bases.

Authors

Ashley P. Tsang, Evan Liu, Conrad Foo, Bob Chen, Hassaan Maan, Donny Chan, Elisa Penna, Veronica Ibarra Lopez, Hector Corrada Bravo, Aicha BenTaieb, Lisa M. McGinnis, Runmin Wei, Kenneth T. Gao

Abstract

Predicting gene expression directly from hematoxylin and eosin (H&E) images offers a scalable and cost-effective approach to deriving spatial molecular insights from routine histology. Pathology foundation models, typically pretrained using tile-level objectives, may not adequately capture the fine-grained cellular features required for single-cell molecular prediction. In this work, we systematically evaluate and establish domain-informed and task-specific strategies for adapting pathology foundation models to single-cell spatial gene expression prediction. Using paired H&E and Xenium 5K data, our approach improves mean gene-wise Pearson correlation by 2.67x over the standard zero-shot baseline and achieves state-of-the-art performance on this task. We demonstrate strong gene-detection performance across the full gene panel (AUROC = 0.800) and importantly identify that predictive performance varies substantially across biological contexts, with our model achieving its highest Pearson correlation for spatially variable genes (r = 0.643). Our findings demonstrate that, despite the challenges of sparse single-cell measurements, H&E-derived representations can recover biologically meaningful spatial expression patterns, highlighting both the opportunities and limitations of morphology-based molecular inference at cellular resolution.