EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address a problem in audio-driven video generation where the process is slow due to repeated image refinement steps. They find that previous methods don't fully use how audio changes over time to make video generation faster. To fix this, they created EchoCache, which uses audio energy patterns to smartly store and reuse parts of the video generation process, saving time and memory. Their tests show that EchoCache speeds up video creation while keeping the video quality and audio matching intact.
audio-driven video generationdiffusion modelsiterative denoisingcross-modal alignmenttemporal-semantic misalignmentenergy-guided cachinglatency-quality trade-offtime-frequency energydynamic timestep-latent cachingquantized cache management
Authors
Jiayu Chen, Xiaoyu Wu, Rongshan Gao, Maoliang Li, Zihao Zheng, Xinhao Sun, Hailong Zou, Guojie Luo, Xiang Chen
Abstract
Audio-driven video generation (A2V) has achieved promising progress in synthesizing temporally coherent and audio-visually aligned videos, yet its inference remains expensive due to the iterative denoising process of diffusion models. Existing caching methods mainly exploit temporal redundancy in visual features while overlooking the cross-modal alignment of A2V, where audio drives visual generation with highly non-uniform temporal importance. In this paper, we identify two levels of misalignment in existing A2V caching methods: temporal-semantic and computation-storage misalignment. To address them, we propose EchoCache, an energy-guided cross-modal caching framework for efficient A2V generation. EchoCache leverages audio time-frequency energy as a saliency anchor to guide latent-level cache updates and further introduces a dynamic timestep-latent caching mechanism with quantized cache management for joint efficiency and memory optimization. Extensive experiments on mainstream A2V models show that EchoCache consistently improves the latency-quality trade-off while preserving generation quality and audio-visual consistency. In particular, on Wan2.2-S2V over the EMTD benchmark, EchoCache achieves a 2.46x speedup with the best overall performance. Code is available at https://github.com/IF-LAB-PKU/EchoCache.