Symbolic formulas speed up neural likelihood inference by more than 100 times

Recovery-Directed Symbolic Distillation of Neural Likelihoods

Machine Learning

Summary

Some models need complicated math to figure out how likely certain outcomes are, but this math can be slow and hard to understand. The authors show how to turn these complicated neural network calculations into simple, clear formulas that are much faster to use. They make sure these formulas still help find model parameters accurately. In tests with a known model, their method speeds up calculations a lot while keeping good accuracy.

What this means in practice

  • For cognitive modelers: Speed up parameter estimation in cognitive models by converting neural likelihoods into fast symbolic formulas without much loss of accuracy.
  • For bayesian data analysts: Accelerate hierarchical Bayesian inference on empirical data by replacing neural likelihoods with efficient symbolic expressions.

Authors

Kianté Fernandez, Xinwei Li

Abstract

Amortized neural likelihoods enable computationally expensive inference for models with analytically intractable or unspecified likelihoods, but their black-box nature limits interpretability. We introduce a symbolic distillation pipeline that converts trained neural likelihoods into explicit, interpretable expressions optimized for efficient parameter estimation. Our approach uses a recovery-directed objective to guide symbolic regression toward expressions that preserve parameter-recovery accuracy rather than merely approximating the likelihood function. Candidate expressions are evaluated on held-out datasets and selected using a criterion that jointly accounts for expression complexity, parameter-recovery performance, and distributional distance from the learned likelihood. We evaluate the pipeline on the diffusion decision model, a classical cognitive model, whose analytically tractable likelihood provides ground truth for controlled evaluation. The proposed recovery-directed objective improves parameter recovery over standard symbolic-regression objectives. The resulting symbolic likelihoods enable over 100 times faster parameter evaluation than both neural likelihoods and, when available, the exact likelihood, while maintaining a manageable loss in precision. We further demonstrate these computational benefits in Bayesian hierarchical inference on empirical data. Our pipeline provides a lightweight interface for integrating symbolic distillation with existing neural-likelihood estimation methods and can be adapted to a range of simulation-based inference settings.