Velocity correction improves complex image generation with minimal slowdown
A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation
Computer Vision and Pattern Recognition
Summary
Text-to-image AI models can make great pictures quickly but sometimes get confused with complicated descriptions involving many objects and their relationships. The authors designed a small add-on tool that fixes these mistakes without changing the original AI or making it slower. This tool learns to adjust the image process step-by-step, helping the AI follow prompts better while keeping the same speed. It works on a frozen big model and improves how well the images match complex instructions.
What this means in practice
- •For digital artists: Create more accurate images from complex text prompts without slowing down the generation process.
- •For mobile app developers: Integrate improved text-to-image generation in apps that require fast and precise rendering from detailed user requests.
Authors
Yating Gao, Zhenlong Zhu, Wen Zihao, Lenan Zhang, Jiexiong Wang, Yang Yang, Guanjun Jiang, Yixiong Zou, Takuya Maekawa
Abstract
Few-step text-to-image generators produce high-quality images with few network forward passes, yet struggle with complex prompts involving multiple subjects, interactions, spatial relations, counting, and attribute binding. Can these errors be reduced without changing the generator's parameters or increasing its forward-pass count? We propose a lightweight correction framework for a frozen 6B-parameter generator. The generator predicts velocities that determine how its intermediate image representation is updated; a corrector with fewer than 20M parameters learns residual adjustments at selected sampling steps. Because earlier corrections change the states encountered at later steps, our On-Trajectory Flow-Bridge Supervision derives residual targets at the states actually reached during corrected generation. A single corrector is shared across selected steps and runs in parallel with the generator. Only the corrector is trained, requiring neither reward optimization nor re-distillation. To our knowledge, this is the first framework for lightweight velocity-residual adaptation of fully frozen few-step text-to-image generators that preserves their original sampling schedule and forward-pass count. On our target complex-scene distribution, the framework achieves a +17.60-percentage-point NetWin over frozen Z-Image-Turbo in pairwise instruction-following evaluation, with only 26.9 ms (0.84%) additional end-to-end latency per image and no increase in backbone evaluations.