Drug target affinity models change with chemical and biological shifts
Beyond Random Splits: Evaluating Drug-Target Affinity Models Under Chemically and Biologically Motivated Distribution Shifts Copy
Machine Learning
Summary
Predicting how well drugs will bind to proteins helps scientists find promising medicines faster. The authors show that the way these predictions are tested can change which computer methods look best. They found that models perform differently when tested on new kinds of drugs or new protein targets, and the best method depends on these testing conditions. This means evaluation should match real-world use to pick the right model.
What this means in practice
- •For pharmaceutical data teams: Improve drug screening pipelines by choosing affinity prediction models tested against real deployment conditions for better candidate prioritization.
- •For bioinformatics software engineers: Develop drug-target affinity tools that incorporate evaluation strategies matching expected chemical and protein shifts for robust model selection.
Authors
Minjae Chung, Clara Li, Malar Paavai Muthukumaran, Shaunna Wang, Aniket Ramkrishnan Iyer, Shaun Qien Yeau Tan, Harinishree Sathu, Micky C. Nnamdi, J. Ben Tamo, Benoit Louis Marteau, May Dongmei Wang
Abstract
Drug-target affinity (DTA) prediction is widely used to prioritize candidate compounds before costly experimental screening. DTA models are often compared under a single data split, even though deployment may require extrapolation to new chemical series, new protein targets, or both. We ask whether the distribution shift used for evaluation changes which architecture appears best. We curate 718,800 unique drug-protein pairs from the ChEMBL and BindingDB datasets. We compare a Morgan-fingerprint + protein-CNN baseline with 12 controlled architectures that combine four drug representations with three ESM-2 interaction modes. Mean validation RMSE increases from 0.950 and 0.945 under scaffold and fingerprint-cluster OOD to 1.299 and 1.321 under protein-cluster and dual OOD. Model rankings are similar across the two chemical shifts (tau = 0.79), but agreement with scaffold OOD falls under protein OOD (tau = 0.39) and reverses under dual OOD (tau = -0.55). Held-out evaluation, repeated seeds, group-aware bootstrap analysis, and a size-matched control support the same conclusion: architecture selection depends on the form of extrapolation, not only on average error or training-set size. DTA benchmarks should therefore match the chemical and target shifts expected at deployment.