CrossDistill improves diffusion model speed while keeping variety and detail
CrossDistill: Balancing Quality and Diversity via Trajectory-Level Hybrid Few-Step Distillation
Computer Vision and Pattern Recognition
Summary
Diffusion models are used to create images and videos but can be slow to generate high-quality results. The authors found that focusing on different stages of the generation process can help: early steps handle broad patterns and later steps add fine details. They created CrossDistill, a method that uses two different training goals for these stages to maintain both variety and clarity in fewer steps. This approach helps speed up the generation without losing important visual features.
What this means in practice
- •For video content creators: Generate diverse and sharp video frames faster from text or images using improved few-step diffusion model distillation.
- •For graphics software developers: Integrate efficient diffusion model acceleration techniques that preserve both detailed quality and variation in generated media.
Authors
Yuxi Liu, Haoyu Li, Yixiang Cai, Tengxu Sun, Zekun Zhang, Baole Ai, Ang Wang, Jiamang Wang, Lin Qu, Kun Yuan, Kai Zhang
Abstract
Few-step distillation accelerates diffusion models but must balance diversity and fidelity: trajectory-based distillation preserves mode coverage, while distribution matching sharpens samples but can reduce diversity. We show that this tension can be exploited in a noise-regime-dependent way: high-noise steps largely determine global modes, whereas low-noise steps refine local details. We propose CrossDistill, a trajectory-level hybrid distillation framework that splits the sampling trajectory at a crossover point, applies a trajectory-preserving objective on the high-noise interval and a distribution-matching objective on the low-noise interval, and couples the two stages through the crossover state. In contrast to loss-level mixing, and complementarily to training-time two-stage recipes, CrossDistill explicitly assigns complementary objectives along the noise axis, so that global branching is preserved before local statistics are sharpened. CrossDistill is a noise-level scheduling policy: PCM and DMD are plug-in instantiations, while the noise partition, crossover coupling, and objective ordering are the key design elements. Experiments on text-to-video diffusion models and qualitative image-to-video results show that CrossDistill expands the few-step quality-diversity frontier, retaining seed-level variation while achieving competitive visual fidelity.