Symbolic regression improved by linking expression parts to their behaviors
SNIP++: Fine-Grained Symbolic-Numerical Alignment for Symbolic Regression
Neural and Evolutionary ComputingArtificial IntelligenceMachine Learning
Summary
Symbolic regression tries to find math formulas that explain observed numbers. The paper’s authors point out that previous AI models link whole formulas to their outputs but don’t understand which parts of a formula cause which effects. They build a new method that breaks down formulas into smaller pieces, linking each to its own behavior before combining them. This lets the model better see how small changes in formulas change the results, helping it work more accurately.
What this means in practice
- •For data scientists: Improve symbolic regression tools to generate more accurate mathematical models by understanding how local changes affect behavior.
- •For mathematical software developers: Enhance expression editing features to reflect how parts of a formula impact its output, aiding interactive model design.
Authors
Benjamin Léger, Shubham Gupta, Samy Mammeri, Kazem Meidani, Cem Subakan, Christian Gagné
Abstract
Mathematical expressions and the numerical behavior they produce are two views of the same underlying function, and connecting them is central to scientific discovery. Symbolic Regression (SR) relies on this connection directly: it searches for an expression that reproduces a given behavior. Recent multi-modal models learn this connection by embedding symbolic expressions and their numerical behavior in a shared representation space. We show that this embedding space is only globally aligned: complete expressions correspond to complete behaviors, but the contribution of individual parts of an expression is not represented. This granularity gap leaves the model unable to tell how a local edit to an expression changes its behavior, the central operation in SR. We introduce a compositional alignment method that closes this gap: a structural positional encoding exposes the substructure of an expression to the encoder, and a multi-granularity contrastive objective grounds each subexpression in the behavior it produces before propagating this grounding to the full expression. The resulting representations close much of the modality gap between symbolic and numerical embeddings, reliably distinguish the effects of local edits that the original alignment cannot, and transfer to external SR corpora.