Summary
Video generation and world models use a memory part called the KV cache, which can be large and slow to use. The authors found that existing methods to shrink this memory can cause flickering and reduce video quality because small errors in the Key part of the memory mess up how the model pays attention to different picture parts over time. They created QuantWM, a new way to compress the KV cache to just 2 bits while keeping video quality steady by carefully picking what details to save and fixing errors that affect attention. Their approach works without retraining the model and makes videos look better and smoother while using much less memory.
What this means in practice
- •For video generation engineers: Compress KV cache memory in video models to reduce resource use while maintaining output quality and temporal smoothness.
- •For mobile device developers: Enable efficient on-device video generation by cutting down KV cache size with minimal quality loss and flicker reduction.
Authors
Jiaqi Zhao, Xiaobin Hu, Bo Yin, Junpeng Jiang, Miao Zhang, Shuicheng Yan
Abstract
KV cache memory has become a major deployment bottleneck for video generation and world models, which motivates low-bit quantization study for efficiency. Existing 2-bit KV cache quantization methods can achieve nearly lossless performance on video benchmarks such as VBench, however, we find that they still cause severe temporal flickering and visual degradation. Meanwhile, deeper investigates show that Key quantization produces smaller reconstruction errors than Value, but surprisingly leads to much larger output degradation. We trace this discrepancy to attention: small Key perturbations can change the attention logits, i.e., QK^\top, and shift the temporal-spatial tokens selected by Queries. These observations motivate us to explicitly preserve attention logits and temporal-spatial token selection during KV cache quantization to alleviate the visual degradation problem. To address this issue, we present QuantWM, a training-free and strictly causal 2-bit KV cache quantization framework. QuantWM introduces two complementary techniques to mitigate the attention shifts. Firstly, quantization-sensitivity-aware clustering (QSAC) jointly considers historical Query sensitivity and residual ranges to select INT2-friendly Key centroids, which reduces quantization errors in channels that are more critical to attention. In addition, principal-subspace attention compensation (PSAC) restores the remaining Key errors along the dominant Query subspace using low-rank projections, which provides a direct and efficient correction to stabilize attention logits. Extensive experiments on Causal-Forcing, LingBot-World-v2, HY-World 1.5, Matrix-Game-2 and Longcat-Video demonstrate that QuantWM significantly improves visual quality and temporal consistency, while outperforming existing methods across image and video quality metrics with up to 6.20x KV cache memory compression and limited additional overhead.