BAG: Budget-Aware Gating for Diffusion Caching

2026-08-10Computer Vision and Pattern Recognition

Computer Vision and Pattern Recognition
AI summary

The authors address a problem in speeding up Diffusion Transformers, which are models used in generating data step-by-step. They propose BAG, a new method that smartly decides when to reuse previous calculations or do fresh ones by considering the overall processing budget and the current progress. This approach combines the advantages of planned schedules and on-the-fly decisions without relying on fixed rules. Their experiments show BAG works better and adapts well to different settings compared to existing methods.

Diffusion TransformersDiffusion cachingfeature reusecomputational budgetgating networkoffline-to-online distillationdynamic schedulingspeedupdenoising steps
Authors
Tong Zhao, Mingkun Lei, Yucheng Han, Chi Zhang
Abstract
Diffusion caching is a lightweight strategy that accelerates Diffusion Transformers (DiTs) by reusing intermediate features across denoising steps, but existing paradigms face a fundamental trade-off: online heuristics lack global budget awareness, whereas static schedules lack instance adaptivity and fail to flexibly adapt to varying runtime budget constraints. To bridge this gap, we present BAG (Budget-Aware Gating), a novel caching policy that unifies global budget pacing with dynamic, instance-adaptive feature reuse. Rather than relying on hand-crafted rules, BAG employs a lightweight gating network that dynamically decides whether to execute a full computation or reuse cached features at each step by jointly conditioning on the budget state and local trajectory feedback. We train this policy via offline-to-online schedule distillation, transferring the decision-making of offline-searched schedules into a compact online gate. Extensive experiments on FLUX.1-dev and Wan2.1 demonstrate that BAG consistently outperforms state-of-the-art caching methods across various speedup tiers while remaining robust across different resolutions, seeds, and guidance scales. Code will be released.