Summary
Sometimes an AI model can get a number right but fail to predict how that number will change if conditions change. The authors develop tests to check if AI models are valid for specific scientific measurements, linking errors and tolerances to the model’s parameters. They show these tests can detect when a model’s predictions about energy and its response differ and improve accuracy in neural network fits. Their approach also helps identify hidden causes behind seemingly incorrect predictions when data is noisy. These methods offer reliable checks on AI models used in science to make sure they truly understand the behavior they are modeling.
What this means in practice
- •For scientific software developers: Add rigorous tests for AI model validity focused on energy and response accuracy in physics simulations.
- •For machine learning engineers: Improve neural network training by enforcing dimensional consistency to drastically reduce prediction errors in scientific domains.
Abstract
An approximation can reproduce an energy while missing its response, or satisfy an error tolerance without being certified by a given bound. We formulate observable-specific validity records for scientific artificial intelligence (AI) that link a reduction to its reference model, error measure, tolerance, parameter domain, and transformations or interventions. Explicit displacement and velocity certificates for a singularly perturbed oscillator recover distinct convergence orders and provide sufficient validity guarantees. Exact Hubbard dimer relations resolve the different coupling domains accepted by energy and response criteria. Across 16 repeated neural fits, imposing dimensional homogeneity reduces energy scale transfer error from 52.3% to 0.0768% and raises the median fraction jointly satisfying gap and response tolerances from zero to 100%. Paired transformations verify the protected scaling relation, while a coupling crossover identifies the additional information needed for response transfer. A causal model recovers a positive intervention response beneath a strong negative observational association. For two mechanisms observed with Gaussian noise whose responses first differ at order $x^k$, we derive an exact fixed control query requirement scaling as $x^{-2k}$ and verify its full success curve numerically. Together with a 256-task evaluation interface, these results provide independently checkable tests of controlled approximation, physical transfer, and mechanism discrimination in scientific AI.