Clinical machine learning models face hidden data challenges from preprocessing choices
Hidden Data Challenges in Clinical Machine Learning
Machine Learning
Summary
When computers learn from electronic health records, the way the data is prepared can hide important details. The authors show that common data cleaning steps can mix up when information was recorded, what health events are labeled, and what missing data means. These hidden issues can change what the computer learns without changing how well it seems to perform. Their work highlights better ways to prepare health data so predictions are more reliable and understandable.
What this means in practice
- •For hospital data teams: Improve data preprocessing pipelines for electronic health records to avoid hidden data leaks and bias in machine learning models.
- •For clinical software developers: Develop machine learning tools that produce clinically interpretable predictions by aligning data processing with medical episode and timestamp semantics.
Tested on one dataset.
Authors
Carolin Cissée, Dennis Fast, Raneen Younis, Alexander Löser, Wolfgang Nejdl, Zahra Ahmadi
Abstract
Widely reused preprocessing pipelines for electronic health records can encode assumptions about when information became available, what clinical episode a label represents, and what missingness means. These assumptions are rarely audited systematically and may propagate across subsequent studies when preprocessing pipelines are reused or adapted. Using MIMIC-IV as a case study, we identify and quantify violations of three complementary forms of preprocessing alignment, i.e., temporal, semantic, and population alignment, which manifest as temporal leakage, feature-task misalignment, and biased imputation. We show how ambiguities in timestamp semantics, episode definitions, endpoint construction, modality selection, and missingness handling can alter the information available to a model and affect the interpretation of its predictions. Through analyses of MIMIC-IV data and empirical case studies, we demonstrate that such choices may remain largely invisible in aggregate predictive performance while changing the signals on which models rely. We derive practical recommendations for temporally valid, clinically aligned, and reproducible preprocessing of MIMIC-IV and related electronic health record datasets. The code for our experiments, including our proposed preprocessing, is available at: https://github.com/m2l-lab/mimic-iv-preprocess.