Elliptic schedules improve sampling efficiency for complex distributions
Elliptic Sampling: Optimal Interpolant Schedules for Gaussians and Beyond
Artificial IntelligenceMachine Learning
Summary
Sampling from complicated probability distributions is important for many AI tasks but can be slow and tricky. The authors propose a new method called elliptic sampling that uses advanced math to better schedule how sampling moves from a starting point to a target distribution. This method is especially good for Gaussian targets and log-concave distributions, making the process more efficient and accurate. They show both theoretical advantages and practical benefits on various datasets, improving over older approaches.
What this means in practice
- •For machine learning engineers: Enhance the efficiency and accuracy of probabilistic sampling in generative models for better training and inference.
- •For statistical simulation teams: Use elliptic schedules to reduce computational costs when sampling complex high-dimensional distributions in simulation workflows.
Authors
Cole Becker, Taylan Kargin, Kimia Hassibi, Asuman Ozdaglar, Pablo A. Parrilo
Abstract
Generative modeling based on stochastic interpolants requires specifying two key choices: an interpolant schedule bridging the initial and target distributions, and an efficient discretization of the resulting probability-flow ODE. In this work, we introduce a new framework for joint schedule/discretization design based on the analyticity properties of the (complexified) ODE trajectories, which in turn ensure good numerical behavior. Our formulation can be solved exactly via conformal mapping, yielding a natural family of variance-preserving elliptic schedules, parametrized by Jacobi elliptic functions. Pairing an elliptic schedule with a tailored discretization, we obtain Zoloflow, a new sampling algorithm closely connected to Zolotarev's optimal rational function approximants. Crucially, both the elliptic schedule and Zoloflow explicitly incorporate the condition number $κ$ of the target distribution and reduce to the classical trig schedule in the limit $κ\rightarrow 1$. For Gaussian targets, Zoloflow achieves minimax optimality among all oblivious score-span algorithms, attaining total-variation error $\varepsilon$ using $Θ(\logκ\,\log(\sqrt d/\varepsilon))$ (exact) score queries. For the log-concave case, the elliptic schedule allows sampling with at most $\varepsilon$ 2-Wasserstein-error by requiring merely $\widetilde{O}(\sqrt{d} \, \mathrm{polylog}(κ, 1/\varepsilon))$ score evaluations. Complementing these theoretical results, we empirically evaluate the elliptic schedule on general (non-log-concave) target distributions, including detailed comparisons with alternative schedule proposals on both synthetic benchmarks and existing image generation datasets.