Single-cell model learns better with biology knowledge than more data
Towards a knowledge-enhanced single-cell foundation model
Artificial Intelligence
Summary
Getting computers to understand the details of individual cells helps biology research, but feeding them more data isn’t always the best way to improve. The authors found that adding biological knowledge about cells and how genes work helps the computer learn better and faster. They built a model called scKITE that uses extra information during its training but keeps things simple afterward. Despite using much less data than earlier models, scKITE does better across many cell-analysis tasks, showing that smart use of biological facts can improve these models.
What this means in practice
- •For bioinformatics teams: Build cell data analysis tools with better accuracy using a biologically informed pretrained model needing far less training data.
- •For drug discovery teams: Improve single-cell based target identification by using models that integrate regulatory gene information for enhanced biological insight.
Authors
Hanqing Zhang, Jie Bao, Mei Ma, Shuai Liu, Jiaying Ma, Jiaguan Liu, Jiaxiao Li, Zhenbo Li, Wenwen Gong, Zhijun Ca
Abstract
Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation and gene-level regulatory information, provided additional scaling dimension than simply increasing data size. Motivated by this observation, we present scKITE, a simple yet effective scFM that integrates cell-annotation and gene-regulatory supervision into a shared transcriptomic Transformer encoder through lightweight auxiliary decoders. These decoders are used only during pretraining and subsequently discarded, yielding a general-purpose encoder enriched with biological knowledge for downstream applications. With only 179,067 pretraining samples, i.e., less than 0.5\% of those used by previous strong scFMs, scKITE outperformed these models across diverse downstream tasks, highlighting knowledge-enhanced pretraining as a promising paradigm for biologically grounded scFMs.