Transformer method helps find scientific equations from data
MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery
Machine Learning
Summary
Finding simple equations that explain scientific data is hard because there are many possible formulas to check. The authors created MOSAIC-SR, a system that uses a trained Transformer model to suggest good starting points for looking for these formulas. Then, it carefully improves the formulas and adjusts constants to better match the data. This method finds correct equations more often than others and works well even when extra unrelated data is present.
What this means in practice
- •For data scientists: Recover interpretable mathematical models from experimental data more effectively without manual formula design.
- •For software developers: Integrate symbolic regression tools that start from learned guesses to improve speed and accuracy in formula discovery applications.
Authors
Peiyi Zheng, Yanming Kang, Hans De Sterck, Giang Tran
Abstract
Symbolic regression aims to recover closed-form equations from observations, providing interpretable models for scientific discovery. Existing approaches struggle to combine flexible structural search with efficient inference. Search-based methods can refine expression structure but often rely on costly combinatorial optimization with random initialization. Pretrained neural models generate formulas almost instantly, but their predictions often contain symbolic errors. We introduce MOSAIC-SR, which uses a pretrained Transformer to propose multiple initial sketches. These sketches initialize searches in several promising regions, avoiding random starts in the vast expression space. Each search jointly recovers structure and constants through scale-aware constant optimization and local symbolic repair. We evaluate MOSAIC-SR on the SRSD-Feynman dataset with and without dummy variables and on six additional benchmarks. MOSAIC-SR obtains the highest symbolic solution rate on every dataset while ranking among the top two methods in predictive accuracy. This advantage persists in the presence of irrelevant dummy inputs. The results show that learned priors can focus search on promising equation structures, and that numerical optimization and symbolic repair are important for recovery.