Latent diffusion gan enhances image sharpness without extra networks
Latent Diffusion-GAN: Adversarial Learning in the Autoencoded Latent Space
Computer Vision and Pattern RecognitionMachine Learning
Summary
Generating high-quality images by gradually removing noise often results in blurry or smooth details. The authors propose a new method that combines two approaches, diffusion models and adversarial learning, without adding extra complex parts. They use a pretrained autoencoder as part of the process to keep images sharp and detailed while maintaining efficient training. This method improves the visual quality and variety of generated images compared to previous techniques.
What this means in practice
- •For computer vision engineers: Create sharper and more detailed images in generative AI tools using adversarial training without adding new network structures.
- •For digital artists and content creators: Generate high-fidelity images with better texture and sharpness for creative tools that need efficient and stable training.
Authors
U-Chae Jun, Jaeeun Ko, Jiwoo Kang
Abstract
Diffusion models are powerful generative frameworks for producing high-quality images by denoising latent variables from random noise. However, training with likelihood-based objectives, such as denoising score matching, can lead to locally over-smoothed high-frequency details, including fine textures and sharp edges, thereby limiting perceptual fidelity and structural detail. Adversarial training with GANs enhances sharpness but typically requires additional discriminator networks, increasing computational costs and destabilizing training. To this end, we propose Latent Diffusion Generative Adversarial Networks (LD-GAN), a novel framework that seamlessly integrates adversarial learning into diffusion models without modifying their original pipeline. LD-GAN leverages the pretrained variational autoencoder (VAE) in latent diffusion models as an energy-based discriminator, enabling adversarial training without extra parameters and preserving the structured latent priors learned from large datasets. We also introduce a structural consistency energy that aligns encoder and decoder feature representations, thereby enhancing perceptual quality and compatibility with the pretrained latent space. Extensive experiments demonstrate that LD-GAN significantly improves sample fidelity, perceptual sharpness, and diversity over state-of-the-art baseline methods across various generation tasks while ensuring efficient training dynamics.