Generative models learn clean images from noisy observations
Learning Clean Generative Processes from Noisy Measurements
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
The paper addresses the problem of learning to create clean images when only noisy versions are available. The authors show that it’s possible to train models to predict clearer images by using pairs of noisy images. To get predictions that are better than the noise level in the training data, they impose a consistency rule that helps refine the predictions to approach clean images. They test this idea on several types of noises and image datasets, demonstrating that the method improves the quality of generated images even from noisy inputs.
What this means in practice
- •For medical imaging teams: Improve image reconstruction in photon-counting microscopy by learning clear images from noisy photon-limited measurements.
- •For remote sensing engineers: Generate clearer synthetic aperture radar images by training on noisy radar data to enhance downstream analysis.
Authors
Fabian Kruse, Lorenzo Baldassari, Ivan Dokmanić
Abstract
We study how to learn a generative model of clean images using only noisy observations. Diffusion processes and their generalizations to non-Gaussian noise require posterior predictions at every noise level. We first show that it is possible to learn these predictions from independent noisy views using Noise2Noise regression, but only at or above the noise level of the training data. To learn the predictions below that level, needed to generate clean images, we impose consistency: a prediction should equal its expectation after continuing the sampling process. We derive conditions under which consistency uniquely determines the continuation to the clean image distribution and verify them for Poisson, binomial, Gamma, and negative-binomial measurements. Experiments on CIFAR-10, photon-counting microscopy, and synthetic aperture radar show that the required posterior predictions can be learned from noisy views alone and that the consistency relation repairs the continuation beyond the noise level of the data, which severely deteriorates without consistency.