Reducing false discoveries in single-cell gene program analysis
Reducing double-dipping in single-cell topic models with three-way count splitting
Machine Learning
Summary
Scientists use topic models to find gene activity patterns in single-cell data, then test which genes link to these patterns using the same data. This causes a problem called double-dipping, leading to many false positive results. The authors tested a new method that splits the data into three parts to separate learning and testing, greatly reducing false positives. Their approach works with different models and improves reliability without claiming better prediction or clustering.
What this means in practice
- •For bioinformatics teams: Implement three-way count splitting to reduce false positive gene associations in single-cell topic modeling workflows.
- •For clinical data analysts: Use donor-level gene program usage to predict disease status more accurately than with cell-type proportions alone.
Authors
Zheng Qu
Abstract
Topic models learn gene programs from single-cell RNA sequencing (scRNA-seq) data, and the same counts are then used to test which genes and donors are associated with those programs. Count splitting is the standard remedy for this reuse, but it has not been tested when an encoder reads each cell's counts. On a 104,400-cell lupus cohort with 300 permuted null genes per run, naive tests rejected 36% of null gene-topic pairs at a nominal 5% level, and the usual two-way split rejected 38%, because the encoder sees the tested gene's own held-out count. Encoding topic usage on a separate third of the counts, with beta-binomial thinning, lowered this to 7.7% (6.7% for genes outside the model); for two other models, LDVAE and scETM, the rate was 7.0-7.3% (naive 51-59%). A label-free interferon program shifts in held-out counts in the lupus cohort and in a pre-specified COVID-19 cohort, and donor-level topic usage predicts disease (AUROC 0.96) beyond cell-type proportions (0.86 and 0.58 in the two cohorts). Calibration and prediction did not depend on the model. We do not claim better clustering, prediction or topic stability. Code, figures and result tables are public (MIT license; DOI 10.5281/zenodo.23307193).