System size affects uncertainty estimates in machine learning atom models
Does System Size Matter in Uncertainty Quantification for Machine Learning Interatomic Potentials?
Machine Learning
Summary
Scientists use machine learning models to predict how atoms interact, and these models estimate how uncertain their predictions are to spot unusual structures. This paper shows that the size of the atomic system changes how these uncertainty scores behave, especially when combining per-atom uncertainties into a single number. The findings reveal that common ways to measure uncertainty don't always work well for larger systems, which can lead to missing unusual atomic environments. The authors suggest calibrating these uncertainty estimates based on system size to better detect new structures.
What this means in practice
- •For computational chemists: Improve detection of unusual atomic arrangements in large simulations by adjusting uncertainty measures for system size.
- •For materials modeling teams: Optimize active learning protocols in machine learning potentials by considering how system size influences uncertainty scores.
Authors
Chayaphol Lortaraprasert, Nontawat Charoenphakdee, Wenwen Li, Yong-Bin Zhuang, Yuta Tsuboi
Abstract
Machine-learning interatomic potentials (MLIPs) commonly use uncertainty estimates to flag out-of-distribution (OOD) configurations and select structures for active learning. A key question is whether the resulting structure-level uncertainty scores remain comparable across different system sizes. A common practice is to aggregate per-atom uncertainties by their mean or maximum and compare the resulting score against a fixed threshold, implicitly assuming such comparability. We test this assumption across several in-distribution (ID)/OOD pairs by characterizing both aggregations as a function of atom count. The two aggregations exhibit distinct size dependencies. Mean aggregation remains size-independent for homogeneous systems but loses sensitivity to defects in large cells. In contrast, maximum aggregation grows with system size for ID systems while remaining approximately flat for defect-containing systems, narrowing the ID-OOD gap in a manner consistent with extreme-value statistics. Although highly uncertain atoms in large structures may indicate novel local environments, retraining on these structures yields little improvement, in contrast to retraining on OOD structures constructed to lie outside the training distribution. Our results reveal a practical limitation of size-unaware uncertainty criteria and motivate size-aware calibration for more reliable OOD detection in large-scale atomistic simulations.