Stable generative models better capture extreme events in data
From Global to Tail Convergence: Dynamics of $α$-Stable Score-Based Generative Models
Machine Learning
Summary
Many models generate data by slowly adding noise, often assuming this noise behaves like normal randomness (Brownian motion). The authors study models that use a different kind of noise with heavy tails, called α-stable noise, which better matches data with extreme or unusual values. They prove that both types of models converge similarly overall, but only the α-stable models accurately reproduce extreme events when the noise type matches the data's tail behavior. This explains why stable models can better capture rare but important data points.
What this means in practice
- •For financial modelers: Improve simulations of rare market events by matching the model noise tails to observed heavy-tailed financial data.
- •For network traffic analysts: Generate synthetic network traffic data that accurately reflects extreme usage spikes by using noise processes with matching tail behavior.
A theory result. No direct application yet.
Authors
Ianis Hammani, Claire Boyer
Abstract
Score-based generative models driven by $α$-stable L{é}vy processes have been proposed as an alternative to Brownian diffusion models, with empirical evidence suggesting improved sampling and better modeling of heavy-tailed data. Yet theoretical guarantees for these models remain limited. We develop a non-asymptotic convergence analysis for scale-preserving $α$-stable generative dynamics, covering both compactly supported and power-law targets. In p-Wasserstein distance, we quantify the effects of early stopping, initialization, score approximation, and time discretization, and show exponential decay of the initialization error. A comparison with Brownian dynamics reveals that global Wasserstein convergence alone does not fundamentally distinguish the two models: both initialization errors decay exponentially with the forward horizon on the common class of heavy-tailed targets. A sharp separation emerges instead in the tails. At any positive generation time, stable dynamics produces power-law tails with index $α$, whereas Brownian dynamics remains light-tailed. For a target with tail index $ν$, asymptotically correct extreme-quantile scaling is possible precisely when $α$=$ν$. These results identify tail-index matching as a principled mechanism behind the ability of stable generative models to capture extreme events.