Improving equation safety by spotting structure uncertainty in AI models
Know When the Equation Is Wrong: Structural Uncertainty for LLM-Guided Symbolic Regression under Extrapolation
Machine LearningArtificial Intelligence
Summary
Finding equations to describe data is useful but tricky when different equations fit the known data but predict very differently outside it. The authors show that current methods can tell when predictions are uncertain within the data seen but often miss it when the model tries to predict beyond that data. They introduced a new method that looks at how much different equation candidates disagree to better judge uncertainty and when to be cautious. Their approach works better at warning about uncertain predictions while still being reliable within the original data range.
What this means in practice
- •For data science teams: Detect when AI-generated equations may be unreliable outside their training data by measuring disagreement between competing models.
- •For scientific computing engineers: Improve the safety of symbolic regression tools in simulations by integrating structural uncertainty estimates.
Authors
Mostafa Cham, Bayu Adhi Tama, Jianwu Wang
Abstract
Large language models (LLMs) can help discover mathematical equations from data, but most LLM-guided symbolic regression methods focus on a single best equation without quantifying uncertainty over alternative functional forms. This is challenging because distinct equations can fit the observed data similarly while producing substantially different extrapolations. Our experiments on the LLM-SRBench dataset show that standard conformal intervals achieve nominal in-distribution coverage but frequently fail under extrapolation. To tackle this challenge, we introduce LLM-SR-UQ, a framework that retains competing equation structures and uses their disagreement to characterize uncertainty about the underlying functional form. We extend the standard fixed-width conformal approach to quantify structural intervals, which measure disagreement among candidate equations and identify where prediction intervals should widen. Extensive experiments show our framework substantially improves coverage attainment by incorporating structural disagreement while preserving in-distribution coverage. Our approach also shows good generalizability when it is applied to other symbolic-regression methods.