Jacobian matrix analysis improves generative image denoiser models
On the spectral properties of generative denoiser Jacobians
Machine LearningComputer Vision and Pattern Recognition
Summary
Generative denoising models create clean images from noisy inputs, but comparing them by final image quality doesn’t reveal how they work differently. The authors look at the Jacobian matrix of the denoiser—the mathematical way it changes output as input varies—to find patterns linked to better image generation. They found that stronger responses along important directions and weaker ones along irrelevant directions lead to better results. By training denoisers to adjust these responses, the models can produce higher quality images.
What this means in practice
- •For image synthesis engineers: Improve image generation quality by tuning denoiser Jacobian spectra during training to balance key data directions and suppress noise.
- •For machine learning platform developers: Incorporate Jacobian spectral regularization tools to enhance pre-trained denoiser models used in image generation services.
Authors
Alexandros Graikos, Nebojsa Jojic, Dimitris Samaras
Abstract
Generative denoising models, such as diffusion and flow-matching, learn to sample from complex distributions by training a deep neural network denoiser to recover clean data from noise-corrupted samples. While such models are typically compared on the quality of their synthesized samples, these metrics provide limited insight into how the underlying denoiser, which drives generation, differs. In this work, we propose to analyze the spectrum of the denoiser Jacobian as a tool to characterize these differences. Across pre-trained denoising models, we observe that better generative performance is associated with larger Jacobian eigenvalues. Motivated by this, we introduce a regularization scheme that controls the Jacobian spectrum by training the denoiser on perturbed inputs, with perturbations suppressing or amplifying Jacobian responses. On ImageNet, we test whether directly modifying the Jacobian spectral properties leads to improved generations. Our findings suggest that denoisers benefit from both strengthening responses along data-relevant principal eigen-directions and suppressing the noisy, data-irrelevant ones. This establishes the denoiser Jacobian as a useful tool for identifying differences between generative denoising models.