Bayesian Symbolic Regression with Entropic Reinforcement Learning
2026-08-10 • Machine Learning
Machine Learning
AI summaryⓘ
The authors address symbolic regression, which is about finding mathematical formulas that explain how inputs relate to an output, especially when data is limited and noisy. Instead of picking just one best formula, they use a Bayesian approach to capture uncertainty and guide the search using prior knowledge. They introduce ERRLESS, a method that uses reinforcement learning to build formulas step-by-step and sample many possible expressions according to their likelihood given the data. Their results show that ERRLESS finds simple, understandable formulas and predicts well, demonstrating the advantages of considering uncertainty in symbolic regression.
Symbolic regressionBayesian methodsReinforcement learningAbstract syntax treesEntropy regularizationPosterior distributionUncertainty quantificationPrior distributionMaximum entropyFeynman benchmark
Authors
Oussama Boussif, Mohammed Mahfoud, Younesse Kaddar, Moksh Jain, Sida Li, Damiano Fornasiere, Xiaoyin Chen, Yoshua Bengio, Esmeralda S. Whitammer
Abstract
Symbolic regression is the problem of finding an algebraic expression describing a stochastic dependence of a target variable on a set of inputs. Unlike forms of regression that fit parameters assuming a fixed model structure, symbolic regression is a search problem over the space of expressions, represented, for example, as abstract syntax trees using a library of operators. Symbolic regression is typically used in settings with limited, noisy data in the natural sciences. However, searching for a single best-fitting expression fails to capture the epistemic uncertainty about the expression, which motivates a Bayesian perspective that enables uncertainty quantification and specification of natural priors to constrain the search space. In this work, we propose ERRLESS (Entropy-Regularized Reinforcement Learning for Expression Structure Sampling), a scalable approach for sampling from the posterior distribution over expressions given data using maximum-entropy reinforcement learning. ERRLESS learns a neural policy that constructs expressions sequentially by building up their abstract syntax trees. At convergence, the policy samples expressions from the posterior. At test time, expressions can be sampled by rollouts of this policy. We demonstrate that ERRLESS achieves competitive results on the Feynman benchmark while producing short and interpretable expressions. Additionally, we demonstrate that the mean of the posterior predictive approximated by ERRLESS achieves a high coefficient of determination ($R^2$) compared to an SMC baseline, highlighting the benefits of the Bayesian perspective in symbolic regression.