Score based method controls generation of rare outlier data points
Score-based Outlier Generation via Controlling the Radon-Nikodym Derivative
Machine Learning
Summary
Outliers are unusual data points that help test how algorithms perform with rare events. Most existing methods don't explicitly control how likely these rare points are. The authors define outliers using math that focuses on low-probability events and show how to change diffusion models to create these outliers without retraining. They test their method and demonstrate it produces rare samples that still fit the overall data structure.
What this means in practice
- •For machine learning engineers: Generate controlled low-likelihood samples to stress-test AI models using existing diffusion models without retraining.
- •For cybersecurity analysts: Create rare but plausible anomalous network traffic patterns to improve intrusion detection systems.
Authors
Amartya Mukherjee, Tristan Milne, Kry Yik-Chau Lui, Stephanie Hazlewood, Jun Liu
Abstract
Outliers are important for stress-testing algorithms and understanding system behaviour under rare conditions. Despite being commonly described as low-likelihood events, existing generative approaches rarely control likelihood explicitly. In this work, we introduce a measure-theoretic notion of outliers based on the distribution of log-likelihood values, which is guaranteed to assign higher probability mass to low-likelihood events with a specifiable magnitude. Building on this formulation, we derive how likelihood reweighting modifies the diffusion score and use this relation to motivate a controlled modification of the reverse-time dynamics. In particular, likelihood reweighting implies a scaling of the score function with a control term derived from the Radon-Nikodym derivative of the likelihood distributions. Correspondingly, the updated score function can be obtained with no retraining of the diffusion model. We exploit the Ornstein-Uhlenbeck semigroup underlying diffusion models to motivate an exponentially interpolated controller which approximates the true control. Experiments demonstrate controlled generation of low-likelihood samples while remaining consistent with the data geometry.