Diffusion models generate diverse high quality images using spectral guided search
Uncovering the Generative Landscape of Diffusion Models: A Spectral-Guided Quality-Diversity Diffusion Framework
Machine LearningArtificial Intelligence
Summary
Diffusion models are good at creating images but usually focus on the most likely results, which limits the variety of images they produce. The paper shows a way to help these models find many different, yet still good, images by combining a method called Quality-Diversity optimization. They also introduce a new technique that guides the search in the model’s space using directions that matter most for the image’s meaning. Their combined approach creates better and more diverse images than many existing methods.
What this means in practice
- •For graphic design teams: Create diverse sets of high-quality images from text prompts improving creative options in visual content workflows.
- •For drug discovery researchers: Generate varied molecular structures with high performance to explore wider chemical spaces during drug design.
Authors
Hannah Janmohamed, Joao Correia, Penousal Machado, Antoine Cully
Abstract
Denoising Diffusion Probabilistic Models have emerged as powerful generative priors across domains ranging from molecular design to high-resolution image synthesis. However, standard diffusion sampling is inherently biased toward high-likelihood regions, often limiting the diversity of outputs. In many domains, a single, high-likelihood sample is insufficient. Instead, diverse portfolios of high-performing solutions are required, motivating methods that uncover a model's full expressive range. Here we show that integrating Quality-Diversity (QD) optimisation with diffusion models enables the discovery of diverse, high-performing outputs. We first propose Quality-Diversity Diffusion, a framework that adapts QD to diffusion models by directly optimising the model's latent noise and conditioning embeddings and leveraging embedding models to quantify diversity. Second, we introduce a novel h-space Spectral Mutation operator, which guides search along directions of maximal semantic sensitivity by analysing principal singular vectors of the model's input-to-activation Jacobian. We evaluate our combined approach, Spectral-Guided Quality-Diversity Diffusion, on text-to-image generation. Against 12 baselines, our method consistently achieves the lowest FID score, demonstrating an excellent balance of image quality and diversity.