Interactive image diffusion optimized for consumer GPUs and speed
The Weight Is Over - Interactive Diffusion on Consumer GPUs
Machine LearningComputer Vision and Pattern RecognitionPerformance
Summary
Generating images with diffusion models usually needs a lot of memory and can be slow, especially on everyday devices like laptops or phones. The authors found ways to make this process faster and use less memory by transforming smaller parts of the model to work like bigger ones, and by carefully balancing speed, quality, and memory use. Their improvements enable people to create and edit images on recent consumer GPUs with almost no wait time.
What this means in practice
- •For mobile app developers: Develop on-device image creation and editing apps that provide sub-second response times on modern consumer GPUs.$Commercial implications: Allows companies to release interactive image generation features for apps running on common consumer hardware with fast user feedback.
- •For graphic design teams: Use an efficient image editing tool based on diffusion models that runs locally on available GPUs for rapid prototyping.
Authors
Frieder Ganz, Maximilian Müller
Abstract
On-device inference is booming, but the momentum is almost all in language models. Diffusion pipelines are memory hungry, latency-sensitive, and require orchestrating an embedder, a transformer, a decoder, and often further postprocessing that is not as standardized as LLM inference loops are. We navigate the trade-off between performance, quality, and model footprint to reach as many client devices in the wild as possible. We make three contributions: an embedding translator that maps a small text encoder into a large encoder space to cut weight and latency; a reproducible sweep recipe for navigating the speed/quality/memory triangle in diffusion pipelines; and an interactive on-device image generation editor achieving sub-second TTFI on recent GPUs.