Gene-aware text and image encoder improves spatial gene expression prediction
GATE-ST: Gene-Aware Text-image Encoder for Spatial Transcriptomics
Artificial Intelligence
Summary
Spatial transcriptomics helps scientists see where genes are active within a tissue by combining gene data with images showing tissue structure. The authors created a method called GATE-ST that uses descriptions of genes written as text to help computers better predict gene activity from tissue images. By mixing information from gene descriptions and tissue pictures, their method matches gene activity more accurately than other approaches. This could save time and money by reducing the need for expensive lab tests.
What this means in practice
- •For pathology labs: Use gene description text combined with tissue images to improve spatial gene expression predictions, reducing reliance on costly experimental tests.
- •For bioinformatics platform developers: Integrate text-based gene information with image data using cross-attention methods to enhance spatial transcriptomics tools.
Authors
Lucas Ni, Jian Luo, Wentao Huang, Chao Chen
Abstract
Spatial transcriptomics enables spatially resolved gene expression analysis from slide-level images while preserving morphological features, providing valuable information for studying disease mechanisms and developing treatments. However, spatial gene expression profiling typically requires expensive and time-consuming tests. While existing image-based prediction optimizations mostly revolve around including positional embeddings and further image-based changes, text-based optimizations remain relatively unexplored. We present GATE-ST, which incorporates text-based inputs into image-based spatial gene expression predictions. With this approach, generated text descriptions of genes are utilized to better spatial transcriptomics prediction results. Gene summaries are put through a text encoder, generating embeddings that integrate with image embeddings through cross-attention layers to align with morphological features. We demonstrate the effectiveness of such text inputs by benchmarking performance against random gene embeddings and multiple other image-text fusion architectures, and show that GATE-ST outperforms these alternatives. Our results demonstrate the effectiveness of GATE-ST in pathology imaging, which may greatly reduce the time and cost of accurate spatial transcriptomic predictions, proving the potential of text-guided spatial gene expression prediction.