Risk-aligned caching speeds up guided image generation with diffusion
RA-CFGCache: From Branch-Level Criteria to Guided-Risk Control under Classifier-Free Guidance
Computer Vision and Pattern Recognition
Summary
Generating high-quality images using diffusion models takes a lot of computing power, especially when using a technique called classifier-free guidance (CFG), which requires extra calculations. The authors point out that current ways to reuse previous work don’t properly handle how errors add up when using CFG or how mistakes at one step affect the final image. They introduce RA-CFGCache, a new method that better estimates risk from reused data and decides when to refresh or reuse parts during image generation. Tests show their approach makes image generation faster without losing quality and works well with different caching techniques.
What this means in practice
- •For visual effects studios: Reduce computation time in guided video or image generation workflows by reusing intermediate results more safely and efficiently.
- •For mobile app developers: Integrate faster high-quality image generation on devices with limited computing resources by better controlling computational reuse under classifier-free guidance.
Authors
Yiming Liu, Ben Wan, Tongxuan Liu, Ao Wang, Yuqi Xiong, Fan Zhang, Hui Chen, Guiguang Ding
Abstract
Diffusion models enable high-quality visual generation, but iterative denoising remains computationally expensive, especially under classifier-free guidance (CFG), which requires both conditional and unconditional evaluations. Training-free caching reduces this cost by reuse of previously computed features or predictions. However, existing branch-local reuse criteria do not explicitly account for how cache errors combine under CFG or how local perturbations affect the final output. We identify two misalignments in cache control: a branch-guided mismatch, where guided error depends on both the magnitudes and alignment of branch errors, and a local-final mismatch, where the downstream impact of a local error varies across timesteps. We propose RA-CFGCache, a Risk-Aligned Caching framework under CFG that incorporates both factors while keeping the sampling schedule and guidance rule fixed. CFG-aware Guided-Risk Composition combines existing branch-wise proxies using CFG coefficients and offline-calibrated cross-branch alignment. Propagation-Aware Rescaling further weights the resulting guided-risk estimate with a timestep-dependent propagation prior calibrated from isolated reuse perturbations. An online threshold controller then determines when to jointly refresh or reuse both branches. Experiments on FLUX.1-dev, Wan2.1-T2V-1.3B, and CogVideoX-2B demonstrate improved efficiency--fidelity trade-offs over evaluated training-free caching baselines. Moreover, RA-CFGCache is compatible with diverse base proxy families, including TeaCache-, DiCache-, and MagCache-style estimators, and consistently improves fidelity at nearly unchanged latency. Code is available at https://github.com/yiming-l21/RA-CFGCache.git.