Multi-fidelity training improves atomistic model accuracy but affects uncertainty calibration
Multi-Fidelity Training Reshapes Uncertainty Decomposition in Atomistic Models
Machine LearningArtificial Intelligence
Summary
When predicting the behavior of molecules, combining lots of simple data with a smaller amount of very detailed data can help make better predictions. The authors studied how this mix changes the model's uncertainty or confidence. They found the approach reduces errors and better ranks predictions but can make the model's certainty misleading, especially when facing unfamiliar molecules or conditions. This means two different ways of measuring uncertainty need checking separately for these models to work well.
What this means in practice
- •For chemical modelers: Use multi-fidelity fine-tuning to reduce prediction errors and improve ranking of uncertainties in molecular simulations across diverse conditions.
- •For materials development teams: Implement refined uncertainty-based acquisition functions from multi-fidelity models to better select candidate materials under varied molecular configurations.
Authors
Matteo Tolloso, Davide Bacciu, Luigi Bonati
Abstract
Multi-fidelity training, which combines abundant low-fidelity data with scarce high-fidelity labels, is increasingly common in atomistic modeling, but its effect on predictive uncertainty remains largely unexplored. We investigate this question using deep ensembles of mean-variance MACE models trained on ANI-1x, contrasting high-fidelity training from scratch with low-fidelity pretraining followed by fine-tuning. We characterize aleatoric, epistemic, and total uncertainty throughout training and under two distribution shifts: unseen molecular systems and higher-energy configurations. We find that pretraining reduces prediction error and improves uncertainty-based ranking across all test sets, but does not yield well-calibrated predictive variances; calibration deteriorates more severely under energy extrapolation. Training-dynamics analysis localizes most of this calibration degradation to high-fidelity optimization, during which the learned aleatoric variance becomes overconfident on held-out data. Moreover, multi-fidelity fine-tuning produces a more effective uncertainty-based acquisition function than high-fidelity training from scratch. These findings show that uncertainty ranking and calibration capture distinct properties and must be assessed separately in multi-fidelity atomistic models, especially under extrapolation.