Summary
Generating long videos with autoregressive video diffusion models is slow because the memory cache grows with the video length. The authors introduce DeCoPrune, a method that automatically identifies parts of the video history that are less important for prediction by comparing intermediate and final clean predictions. This approach keeps important information while allowing pruning of less useful data, making the video generation process faster and more efficient without needing extra training. They also created a new benchmark, CMBench, to test how well methods remember earlier video events, showing DeCoPrune retains important details while speeding up generation.
What this means in practice
- •For video generation engineers: Speed up long video generation by pruning less informative tokens from the autoregressive model's cache while preserving important visual details.
- •For interactive content developers: Enable more efficient real-time video streaming and editing by maintaining essential context in the cache and trimming unnecessary history.
Authors
Zeqi Xiao, Qingle Liu, Kaiwen Zhang, Yifan Zhou, Zihan Ding, Xingang Pan
Abstract
Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows with the generated history. Existing compression strategies discard history using fixed windows or select tokens through local attention and similarity signals, without directly measuring whether a chunk contributes information beyond the retained context. We introduce DeCoPrune, a training-free method that treats cache compression as a denoising-consistency problem. We find that tokens with larger discrepancies between intermediate clean predictions and final denoised values tend to carry visual evidence less predictable from the retained context. DeCoPrune uses this model-intrinsic signal to retain high-discrepancy tokens in the long-term cache while pruning low-discrepancy tokens. To evaluate information retention, we introduce CMBench, comprising 58 approximately one-minute generated or real-world context episodes and 116 Reappear or Revisit continuation tasks requiring recall of earlier events or objects. Experiments with LingBot World v2 show that DeCoPrune achieves a DINO score of 0.6701 on a 0-1 scale, with an 85.43% reduction in cumulative historical KV token counts and a 4.14-fold continuation-generation speedup over FullKV. Its head-specialized variant reaches 0.6783 at an 86.19% pruning ratio, approaching FullKV's 0.6803 score and exceeding the evaluated compression baselines at similar budgets. These results indicate that denoising consistency can support long-range information retention while reducing autoregressive inference cost. Our project homepage is https://decoprune.github.io. The code is available at https://github.com/DeCoPrune/CMBench, and the benchmark at https://huggingface.co/datasets/Aoraku/CMBench.