Video generation memory usage cut by predictive cache compression
TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation
Computer Vision and Pattern Recognition
Summary
Generating long videos with autoregressive models demands a lot of memory due to large cached data. The authors noticed that parts of this cached data change in predictable ways over time, especially keys and values which behave differently. They created TempoKV, a method that predicts these parts and stores only small differences, reducing memory use without hurting video quality. Their tests show TempoKV compresses cache data much more effectively than older methods while keeping video quality high.
What this means in practice
- •For video software engineers: Reduce memory load in autoregressive video generation by compressing the key-value cache efficiently with a predictive method.
- •For ai hardware architects: Design hardware accelerators optimized for predictive KV cache compression to enable longer autoregressive video generation sessions.
Authors
Jiahui Hu, Weile Mo, Peiliang Cai, Jiacheng Liu, Jinkui Ren, Xiantao Zhang, Xiaobing Tu, Linfeng Zhang
Abstract
Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose \textbf{\method}, a training-free predictive KV cache quantization framework for AR video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to $6.71\times$ KV cache compression with the best overall quality--efficiency tradeoff. Our code will be released on GitHub.