Improving rare event probability estimates with safer sampling method
Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures
Machine Learning
Summary
Estimating the chance of rare events, like extreme system failures, can be very difficult because such events happen so infrequently that normal methods need huge amounts of data to get accurate answers. The authors propose a new way to improve these estimates by carefully choosing how to sample data, using a mix of Gaussian distributions with built-in safety checks to avoid unstable results. Their method separates the problem into first finding a reasonable sampling range and then refining it while ensuring stable calculations. Tests show this approach reduces errors and works well even in complex, high-dimensional cases where other methods struggle.
What this means in practice
- •For risk assessment teams: Estimate probabilities of rare but critical failures in complex systems more reliably and efficiently using the new constrained sampling method.
- •For financial modelers: Improve accuracy and robustness of rare-event loss probability estimates in high-dimensional financial portfolios with the proposed importance sampling framework.
Authors
Paweł Lorek, Rafał Nowak, Rafał Topolnicki, Tomasz Trzciński, Maciej Zięba
Abstract
We study estimating rare-event probabilities $I = \mathbb{P}(g(\mathbf{X}) > γ)$ with $\mathbf{X} \sim \mathcal{N}(\boldsymbolμ, \boldsymbolΣ)$ and general $g : \mathbb{R}^d \to \mathbb{R}$. We address this problem through importance sampling, and propose a framework that substantially improves efficiency and robustness over baselines such as crude Monte Carlo, adaptive cross-entropy, variational-inference-based methods (including reverse- and forward-KL approaches), as well as Safe-ICE, Subset Simulation, and Sequential Monte Carlo, drawing on ideas from both rare-event estimation and cross-entropy optimization. The key contribution has two parts: first, we separate the problem into coverage, to overcome the cold-start barrier, and fitting, to refine proposals once a meaningful signal is available; second, we constrain the final GMM proposal so that it has finite importance-sampling variance (since coverage alone is not sufficient -- without safeguards, importance sampling may still suffer from infinite variance). Together, these ingredients yield expressive proposals; finite variance does not by itself guarantee practical stability at a fixed sampling budget. Extensive experiments demonstrate substantial variance reduction, strong robustness across diverse benchmarks, and favorable cost--efficiency trade-offs, with the proposed approach often outperforming these baselines, particularly in high-dimensional and multimodal settings where competing methods frequently become unstable or fail. Our code is available at https://github.com/lorek/robust-cfi-is.