Energy-aware noise improves image diffusion model quality
Energy-Conditioned Noise Schedule and Whitening for Spectral Diffusion
Artificial IntelligenceComputer Vision and Pattern RecognitionEmerging Technologies
Summary
Diffusion models generate images by gradually adding and removing noise. Traditional methods add noise uniformly without considering the energy of different parts in the image's frequency components. The authors introduce a way to adapt how much noise is added based on the energy of each frequency component, making the process more natural and effective. This approach improves image generation quality without changing the core model design.
What this means in practice
- •For image generation developers: Improve the quality of generated images by using an energy-conditioned noise schedule in diffusion models working in frequency domains.
- •For signal processing engineers: Use energy-based noise allocation and spectral whitening techniques to enhance transform-domain noise modeling for image and video processing applications.
Tested on one dataset.
Authors
Bata Vasic, Bane Vasic
Abstract
This paper introduces an energy-adaptive noise scheduling and whitening strategy for transform-domain diffusion models. Existing spectral diffusion methods account for the non-uniform statistics of transform coefficients through coefficient scaling, normalization, or frequency prioritization, while the forward diffusion noise schedule remains largely independent of the underlying spectral-energy distribution. We investigate whether the temporal evolution of the forward diffusion process should also follow the spectral organization of natural images. The proposed formulation combines global spectral whitening with energy-conditioned noise allocation that jointly modulates the injected noise according to the energy of individual transform coefficients and an image-dependent energy path over diffusion time. The resulting forward process preserves Gaussian transitions with closed-form marginals and remains compatible with standard DDPM and DDIM procedures without modifying the diffusion architecture. Experiments on CIFAR-10 demonstrate the contribution of the proposed energy-conditioned noise schedule and spectral whitening, reducing Fréchet Inception Distance from 142.48 for a compact DCTdiff U-Net variant to 100.45.