Attributing disease effects in age prediction models to training data samples
Attributing Cohen's d: Training Data Attribution for Disease-Related Effects in Normative Age Biomarkers
Machine Learning
Summary
Age prediction models learn to estimate a person’s age based on healthy people’s data. When these models are used on patients, the difference between predicted and actual age hints at disease effects. The authors develop a way to find which specific training data samples influence this disease-related age difference the most. Removing the most influential samples surprisingly increases the model’s ability to detect disease effects. They also provide software so others can apply their method.
What this means in practice
- •For clinical data scientists: Improve disease signal detection in age biomarker models by identifying and adjusting influential training samples.
- •For biomarker developers: Enhance biomarker models by removing training samples that obscure disease-related effects to better represent underlying patient biology.
Authors
Jakob Snel, Marc-Andre Schulz
Abstract
Normative age models are trained to predict chronological age in a nominally healthy cohort. Applied to patients, they deviate, and the gap between predicted and chronological age is read as disease risk. Here, we attribute the disease-related effect size of the age gap directly to individual training samples, rather than using a prediction-level loss as the attribution target. For Cohen's $d$, the resulting closed-form influence functional, validated against leave-one-out retraining, ranks training samples by their effect on held-out case-control separation. Across four diseases and two biomarker modalities in UK Biobank, removing the 10% most influential training samples raises held-out disease-related effect size in every seed. It more than doubles the metabolomic-age effect for type-2 diabetes and raises the brain-age effect for multiple sclerosis by roughly a third. Random removal leaves effect size flat even at 50% removal, confirming the gain comes from which samples are removed, not how many. Flagged subjects carry subclinical cardiometabolic burden that diagnosis-based exclusion misses, on markers the model never sees. For type-2 diabetes, where the method gains most, the marker recovered is HbA1c, the standard measure of blood sugar control. We release pyinfluence, our influence-function package, for reproducibility and reuse.