Speed up image editing by freezing unchanged pixels and caching steps
Freeze What Is Settled, Cache What Is Smooth: Spatial Freezing Keeps Step Caching in Its Low-Error Regime
Computer Vision and Pattern RecognitionMachine LearningMultimedia
Summary
Image editing models often take a long time because they redo many small steps to improve images. The authors show how to split the work between two tricks: freezing parts of the image that don't change and reusing saved calculations from previous steps. Their new method, FreezePlan, speeds up editing by over five times with little loss in image quality. This approach helps keep the errors low by carefully balancing how much work is saved by freezing versus caching.
What this means in practice
- •For mobile app developers: Build faster instruction-based image editing features by implementing FreezePlan to reduce processing time with minimal quality loss.
- •For cloud service engineers: Improve cloud-based image editing service throughput and cost efficiency by integrating FreezePlan for accelerated diffusion transformer execution.
Authors
Baiyan Li, Ganxuan Yang
Abstract
Instruction-based image editors built on diffusion transformers run tens of denoising steps. Training-free acceleration either reuses features across steps (step caching) or stops computing the regions an edit leaves untouched (region freezing), and the two have been combined before. We ask how a high speed-up target should be split between them. On Qwen-Image-2.1, freezing does more than remove tokens: it lowers the cost of every step, so the step cache can reach the same overall speed-up with a conservative threshold that keeps its error small. A per-step cost model fitted on development runs predicts held-out latency with a median error of 4.0% and reproduces the speed ceilings of the region methods we re-implemented (2.73$\times$ predicted vs. 2.70$\times$ measured for RegionE). Driving frozen tokens to the model's one-step $x_0$ estimate beats returning them to the reference image on every development sample, and in 90% of measured cases the composed error stays below the sum of the two individual errors. The resulting method, FreezePlan, keeps five full steps and gives the rest of the budget to TaylorSeer. On a pre-registered test split it runs 5.05$\times$ faster end to end at 29.8 dB PSNR to the original output, with no significant drop in a VLM judge score; TaylorSeer at 5.02$\times$ reaches 23.5 dB. The curve extends to 5.99$\times$ at 28.0 dB. Swapping in the region selectors of prior methods under our schedule lowers median PSNR by only 0.5--0.6 dB, which places the gain in the schedule and budget split rather than in the selector.