Attribution analysis reveals meaningful signals in single-cell RNA AI models
Attribution-based diagnostics for single-cell foundation models
Machine Learning
Summary
Single-cell RNA models help scientists understand cells from their gene activity, but it’s unclear if these models rely on real biology or shortcuts. The authors studied how the models make decisions by tracing what genes influenced predictions for identifying cell types and responses to changes. They found attribution methods can show which genes matter, how the model’s understanding evolves, and when it might be making mistakes. Their work demonstrates that these post-hoc techniques help explain model behavior beyond simple accuracy numbers.
What this means in practice
- •For bioinformatics teams: Assess model trustworthiness by identifying biologically meaningful features driving cell type predictions in single-cell analysis pipelines.
- •For drug development teams: Improve understanding of how AI models predict cellular responses to treatments by comparing model attributions with known regulatory pathways.
Authors
Farnoush Rezaei Jafari, Tristan Lazard, Kenza Bouzid, Shruthi Bannur, Jeremiah Wander, Ashley Mae Conard, Stephanie Hyland, Lorin Crawford, Daniel Coelho de Castro
Abstract
Single-cell RNA foundation models (scFMs) are increasingly used for in-silico biological discovery, yet predictive performance alone does not reveal whether their decisions rely on biologically meaningful signals or model-specific shortcuts. We systematically investigate post-hoc attribution of downstream predictions across two tasks: cell-type annotation using pretrained Cell2Sentence and scGPT models, and perturbation prediction using scGPT+STATE in a controlled setting. For cell annotation, we use attribution to assess whether predictions rely on established marker genes and how their relevance varies with model architecture, expression rank, and the surrounding cellular context. For perturbation prediction, we compare attribution patterns with known regulatory ground-truth in a synthetic setting to assess what they reveal about the model's learned computations. Across these tasks, attribution reveals biological feature use, contextual interactions, contributions from different input pathways, learning progression, associations with prediction quality, and potential pitfalls that are not fully captured by standard performance metrics. Experiments with complete and limited regulatory knowledge further demonstrate that conclusions drawn from attribution depend strongly on the choice of biological reference, baseline, and null model. Together, these findings highlight the value of attribution as a flexible post-hoc diagnostic for understanding model behavior, identifying potential issues, and informing the development of scFMs.