Unified video codec enables quality and low-delay decoding from one stream
Unified Generative Video Compression with Autoregressive and Bidirectional Video Diffusion
Computer Vision and Pattern Recognition
Summary
Video compression often must choose between better quality or faster playback. The authors created a single video codec called UniGVC that can do both by supporting two ways to decode video from the same compressed data. One way focuses on high quality but needs the whole video first, while the other prioritizes fast streaming with lower quality. This helps in situations where videos are watched live and then stored for later high-quality replay. The system performs well on standard videos and can decode full HD video quickly or with high visual quality, depending on the mode.
What this means in practice
- •For video streaming engineers: Implement adaptive video streaming solutions that switch between low-latency live play and high-quality stored playback using a single compressed video stream.
- •For mobile app developers: Build mobile video players capable of delivering both fast live video with minimal delay and stored high-quality video without extra bandwidth.
Authors
Tianyu Zhang, Shuai Huo, Dengchao Jin, Hewei Liu, Meng Lei, Tianwen Wu, Jiawen Gu, Chao Zhou, Li Li, Dong Liu
Abstract
Video diffusion models can reconstruct visually compelling content from severely compressed representations at ultra-low bitrates, but they impose a rigid deployment trade-off. Bidirectional (BI) models achieve high quality through dense full-sequence attention, yet require the entire sequence before reconstruction and incur substantial latency. Autoregressive (AR) models enable low-delay streaming, but their restricted temporal context reduces compression efficiency. This tension is particularly relevant when a video stream must be decoded in real time yet is also stored for later, quality-oriented playback, making AR and BI models natural choices for the live and replay stages, respectively. To support both stages, we present UniGVC, a Unified Generative Video Codec that supports two decoding modes from a single bitstream. UniGVC-BI employs a BI-DiT with dense attention for quality-oriented random-access decoding, whereas UniGVC-AR uses windowed AR-DiT for low-delay streaming. Both paths share a hierarchical latent codec that recurrently produces the same conditions with ultra-low bitrates, allowing UniGVC-BI to serve as a conditional teacher for single-step distillation of UniGVC-AR. A tiny conditional video decoder further reduces streaming latency while preserving most of the reconstruction quality. On standard 1080p benchmarks, UniGVC-BI achieves the best overall rate-distortion-perception trade-off among the evaluated methods and produces temporally stable video at rates as low as 0.0005 bits per pixel. Meanwhile, UniGVC-AR remains competitive while decoding 1080p video at 36.85 FPS, over $250\times$ faster than UniGVC-BI. Code and models will be released later.