MixDiffusion: Mixing Diffusion-based Uni-condition Text-to-Image Generation Models for Multi-condition Image Synthesis
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed MixDiffusion, a new method that can create images from text combined with many types of control inputs like sketches or depth maps. Unlike previous methods that handle only one type of control, MixDiffusion merges multiple single-condition models without needing extra training. It uses a mathematical formula to combine their predictions at each step, allowing it to flexibly generate images with multiple guidance conditions. This makes MixDiffusion easy to use and adaptable to new control types.
text-to-image generationdiffusion modelscontrol conditionsnoise predictionmulti-condition generationbounding boxeskeypointsdepth mapstraining-free modelsimage synthesis
Authors
Pengcheng Wan, Liang Han, Lin Xu, Bowen Xiao, Liqiang Nie
Abstract
Recent advances in text-to-image (T2I) generation have enabled controllable image synthesis by incorporating conditions beyond text. However, most existing diffusion-based methods are limited to a single type of control condition (e.g., bounding boxes or keypoints), which restricts their flexibility. To address this limitation, we propose MixDiffusion, a training-free diffusion framework for multi-condition T2I generation. MixDiffusion theoretically supports an arbitrary number of control conditions, including bounding boxes, keypoints, sketches, depth maps, reference images, and text, by collaboratively integrating multiple pre-trained uni-condition diffusion models. The key insight of the proposed approach is to derive the predicted noise distribution in each denoising step of the diffusion-based multi-condition image generation model from the predicted noise distributions of multiple diffusion-based uni-condition models with a derived integration formula, which is supported by rigorous theory proof. Owing to its training-free nature, MixDiffusion is easy to deploy and readily extensible to new control modalities.