Token Radius Attention for Efficient Video Generation
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors study a type of video generation model called Video Diffusion Transformers, which work well but are slow because they look at all parts of the video equally. They found that not all parts need the same attention and that this need relates to a measure called attention entropy. Based on this, they created Token Radius Attention, a way to focus computation only on important nearby tokens without extra training. This method speeds up the model by up to twice as fast while keeping similar quality.
Video Diffusion Transformersself-attentionattention entropysparse attentiontoken radiusvideo generationtransformersblock-sparse mask
Authors
Jiayu Chen, Zhikun Jiang, Maoliang Li, Jiayi Luo, Jiawei Yang, Zihao Zheng, Hengyi Zhang, Guojie Luo, Xiang Chen
Abstract
Video Diffusion Transformers (VDiTs) enable high-fidelity generation but incur quadratic cost from dense 3D self-attention. Existing head- and block-level sparse methods share computation budgets across queries, overlooking token-specific attention demand. We observe that retained density varies across queries yet correlates log-linearly with attention entropy, while dominant interactions form query-centered neighborhoods with token-dependent radii. Based on these findings, we propose Token Radius Attention (TRA), a training-free framework that maps query entropy to an analytic token budget and converts it into a temporally decayed radius without explicit key ranking. Fused entropy extraction, warm-up reuse, and block-sparse mask construction further reduce overhead. Across seven Wan2.1, Wan2.2, and HunyuanVideo T2V/I2V configurations, TRA retains only 9-19% of attention interactions and achieves 1.56x-2.05x speedup with competitive generation quality. Code is available at https://github.com/IF-LAB-PKU/Token-Radius-Attention.