CellRFT uses reinforcement learning to improve cell response predictions
CellRFT: Reinforcement Fine-Tuning for Single-Cell Perturbation Modeling
Machine Learning
Summary
Predicting how cells change after certain treatments is important for understanding biology and medicine. Existing methods try to guess these changes but often train with goals that don't match how scientists actually judge the predictions. The authors present CellRFT, a way to teach models using feedback based directly on biological criteria, even if these are hard to measure mathematically. This helps the models make better predictions that align more closely with what biologists want to see.
What this means in practice
- •For biotech data teams: Improve predictions of cellular responses to genetic or drug perturbations in single-cell data analysis pipelines using biologically guided fine-tuning.
- •For pharmaceutical modelers: Enhance in silico evaluation of drug effects at the single-cell level by integrating multiple biological feedback criteria during model training.
Authors
Jie Yan, Li Liu, Hanze Guo, Jiaxin Hu, Houxin He, Xiaoning Qi, Haoran Wang, Cong Li, Zhong-Yuan Zhang, Yong Wang
Abstract
Predicting cellular responses to perturbations supports the study of gene function, disease mechanisms, and therapeutic strategies. Despite advances in single-cell perturbation modeling, existing models typically optimize surrogate losses that do not directly reflect the biological criteria used for evaluation, so better data fitting need not yield better biological predictions. To address this mismatch, we introduce \textbf{CellRFT}, a reinforcement fine-tuning framework that uses biological evaluation as direct training feedback. CellRFT uses policy-gradient optimization to learn from non-differentiable evaluations of generated cell populations and integrates multiple biological rewards through hierarchical reward aggregation. Comprehensive experiments demonstrate CellRFT's applicability across different pretrained models and effectiveness in improving perturbation prediction, reveal that optimizing one biological criterion can help or hinder others, and show that complementary rewards can improve criteria beyond those directly optimized, offering a way to probe how biological metrics shape model behavior, with the potential to inform evaluation design. Code will be made available.