Diffusion free score matching enables exact conditional sampling
Diffusion-free Score Matching for Exact Conditional Sampling
Machine Learning
Summary
Generating realistic data, like images or sounds, often involves adding noise and learning to reverse this process. Traditional methods rely on multiple noisy steps, which makes it hard to generate data conditioned on specific features. The authors propose a new approach called DISCO that avoids multiple noise levels and learns a simpler, single-step model. This makes it easier and more accurate to generate data based on specific conditions, improving flexibility and reliability in data generation.
What this means in practice
- •For machine learning engineers: Generate data samples that precisely match any specified condition, improving model reliability in conditional generation tasks.
- •For image generation developers: Create images with exact user-defined features by sampling from accurate conditional distributions without complex diffusion steps.
Authors
Thomas Wedenig, Robert Peharz
Abstract
The success of modern score-based generative models is largely driven by diffusion-based training, which learns a collection of time-indexed score fields corresponding to progressively noisier versions of the data distribution. While such models achieve impressive sample quality, leveraging them for sound probabilistic inference---particularly sampling from arbitrary conditionals of the model---remains challenging. This difficulty arises because conditioning information is only available for clean data and not for the higher noise levels required to generate exact conditional samples. In this paper, we introduce DIffusion-free SCOre matching (DISCO), a principled approach that sidesteps the need for time-dependent score fields altogether. Our method is based on an objective that, while reminiscent of diffusion-based training, estimates only a single score of the (slightly perturbed) data distribution, for which the exact conditional score is readily obtained. In experiments, DISCO is competitive with state-of-the-art diffusion models in terms of sample quality. More importantly, DISCO yields a more faithful representation of the underlying data distribution and---crucially---enables accurate sampling from arbitrary conditionals, outperforming standard heuristic samplers. This capability opens the door to principled and flexible probabilistic reasoning with score-based models.