Video DeltaNet speeds up livestream video generation by over 14 times
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Machine Learning
Summary
Making high-quality livestream videos using AI normally takes a lot of computing power because the AI pays close attention to many parts of the video all at once. The authors developed Video DeltaNet, which mixes a faster attention method with a traditional one to keep good detail while speeding up the process. This approach updates video memory once per frame and cleverly balances both methods to reduce the time needed to generate videos. As a result, it can create 14-second high-definition videos much faster using multiple GPUs.
What this means in practice
- •For livestream video platform engineers: Accelerate high-quality livestream video generation to support real-time video content creation and editing.
- •For cloud video rendering teams: Reduce the computing time needed to render long, high-resolution videos in the cloud by optimizing the attention process in video models.
Authors
Haocheng Xi, Yiming Xie, Hexu Zhao, Yiwen Zhang, Michael Liu, Thomas Creavin, Kurt Keutzer, Xiuyu Li, Zhaoyang Lv, Chenfeng Xu, Haiwen Feng
Abstract
Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major computational bottleneck. Linear attention offers an appealing alternative and has been widely adopted in recent large language models, but directly applying it to video models often fails to preserve the fine-grained interactions required for high-quality generation. We present Video DeltaNet (VDN), which combines local Softmax attention with bidirectional linear memory for long-range video context. Its linear branch introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens. Separate output projections and learnable gates calibrate the two branches, while a staged teacher-alignment recipe progressively introduces the new pathway into pretrained models. We instantiate VDN on MiniMax H3, applying the hybrid to video-to-video interactions while retaining Softmax for interactions involving text or audio. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 completes DiT denoising for a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs, corresponding to a 14.5x speedup over the 50-step dense H3 baseline on the same GPU count.