Protein interaction datasets have hidden biases that confuse machine learning
Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
Machine Learning
Summary
Protein-protein interaction databases are supposed to show how proteins work together in living things, but they contain hidden biases from the way data is collected and organized. These biases trick machine learning models into learning shortcuts that don’t reflect true biology. The authors studied several major databases and found multiple such biases that remain even after careful splitting of data. They then created a new open-source method that detects and reduces these biases when building datasets for training models, helping make predictions more reliable.
What this means in practice
- •For bioinformatics engineers: Construct PPI training datasets that minimize misleading shortcuts, improving model reliability for predicting true protein interactions.
- •For data scientists in other fields: Apply the approach of bias-aware negative sampling and dataset splitting to reduce shortcuts in classification problems with large negative candidate sets.
Authors
Judith Bernett, Anton Spannagl, Joel Ås, Markus List, David B. Blumenthal
Abstract
Protein-protein interaction (PPI) databases do not faithfully reflect biological realities. Instead, they are influenced by study and technical biases that distort certain protein and interaction attributes. Machine learning models can exploit these as learning shortcuts if the negative dataset is not constructed with care. So far, the shortcuts introduced during PPI dataset construction have only been examined in isolation. Here, we systematically characterize both reported and, to our knowledge, previously unreported biases in PPI datasets that lead machine learning models to learn shortcuts instead of biological signal. We analyze HIPPIE, IntAct, and STRING, dedicated PPI databases, as well as two datasets derived from 3D-structural information in the Protein Data Bank (PDB). We show that random data splitting introduces strong topological shortcuts. When train-test protein overlap is removed, the resulting datasets still retain usable shortcuts stemming from self-interactions, taxonomic identity, and functional relatedness, whose prevalence interestingly depends on the data source. We further show that sampling negatives from a set of high-confidence non-interactors, an intuitively appealing choice, can amplify the shortcut stemming from functional relatedness. To detect and mitigate these biases, we provide an open Nextflow pipeline that combines similarity-aware, data-loss-minimizing dataset splitting with bias-minimizing negative sampling, both formulated as integer linear programs. Its key concept of quantifying biases to minimize them through optimization-based negative sampling can, in principle, be extended to any machine learning problem where the pool of negative candidates is much larger than the positives and is thus of interest also beyond PPI prediction.