Post-training fix improves extreme value modeling in diffusion models
Radial Tail Repair: Post-Training Correction for Diffusion and Flow Models
Machine Learning
Summary
Modeling rare extreme events is hard for diffusion and flow-based AI models, especially when data has heavy tails, which means extreme values happen more often than usual. The authors propose a simple fix that adjusts the model only in the extreme regions without retraining the whole model. Their method separates the direction and strength of data points and corrects just the strength for the extremes. This improves how well the model matches rare events in tests on synthetic and real weather and wildfire data.
What this means in practice
- •For weather forecasting teams: Improve the accuracy of extreme weather event simulation without retraining models from scratch.
- •For wildfire prediction units: Enhance the modeling of rare extreme wildfire conditions by adjusting pretrained generative models post-training.
Authors
Hamza Cherkaoui, Hélène Halconruy, Antonio Ocello
Abstract
Diffusion and flow-matching models may struggle to accurately represent distributional tails, especially for heavy-tailed data. Existing approaches often use heavy-tailed noise or modify the generative dynamics, requiring dedicated retraining and more difficult estimation. We take a simpler approach: rather than relearning the full distribution, we correct only the model's behavior in the extreme regime. Our method builds on the radial-angular decomposition of multivariate heavy tails, where radius and direction become asymptotically independent. This allows us to preserve the directional structure learned by the pretrained model and correct only its radial tail. We estimate the tail onset from reference and generated samples, then rescale generated extremes so that their radial survival matches the target distribution. The resulting method is a lightweight post-training correction applied to a frozen diffusion or flow-matching model, requiring neither retraining nor changes to its sampling dynamics. Experiments on challenging synthetic heavy-tailed distributions and real weather and wildfire datasets show that this simple correction substantially improves extreme-tail coverage using only a small number of fitting samples.