Residual patterns improve discovery of scientific formulas with AI
RISR: Residual-Informed Scientific Equation Discovery with Large Language Models
Symbolic ComputationArtificial Intelligence
Summary
Finding the right equation to explain scientific data is hard because small errors can hide important details. The authors introduce a method called RISR that looks closely at where current guesses go wrong and uses this information to suggest better formulas. By encoding these errors and guiding a language model, RISR finds formulas that fit data more accurately, even on new types of data. Tests show it works better than previous methods at discovering scientific equations.
What this means in practice
- •For data scientists: Improve model discovery by using residual patterns to find better-fitting equations for complex datasets.
- •For engineering teams: Automate derivation of physical formulas from experimental data, reducing manual trial-and-error.
Authors
Haobo Li, Wenshuo Zhang, Wenxiao Zhao, Eunseo Jung, Rui Sheng, Yushi Sun, Peiqin Zhuang, Hao Chen, Fenghua Ling
Abstract
Symbolic regression combines structural search with numerical fitting, but aggregate fit scores do not describe how the remaining error varies across inputs. We introduce RISR, a residual-informed method that uses these error patterns to guide formula discovery and learn which corrections are worth fitting. A residual encoder compresses aligned inputs, targets, current predictions, and residuals into continuous tokens that condition a language model to propose formulas. For subsequent refinement, a dual-view relational encoder uses additive and regularized multiplicative residuals to predict the post-fit utility of candidate corrections. We evaluate RISR on scientific tasks from the LLM-SRBench. RISR achieves 63.57% and 38.50% ID accuracy at the 1% and 0.1% pointwise relative-error tolerances, respectively. The corresponding OOD accuracies are 56.07% and 38.24%. RISR outperforms the reported baselines using the same backbone. The results show that our residual-informed approach can improve numerical equation recovery.