Residual informed flow transfer improves generative video compression quality
RIFT: Residual-Informed Flow Transfer for One-Step Generative Video Compression
Computer Vision and Pattern Recognition
Summary
Video compression methods sometimes lose fine details, especially when using new AI-based techniques that generate parts of the video during reconstruction. The authors found that existing methods don't perfectly match the changes caused by compression, which reduces quality. They developed a new method called RIFT that uses leftover error information to better adjust these changes, making the generated videos look closer to the original. Experiments showed that RIFT produces better video quality than previous generative compression approaches.
What this means in practice
- •For video streaming service teams: Enhance compressed video reconstruction by integrating residual-based flow adjustments, improving quality without major codec changes.
- •For mobile video app developers: Implement one-step generative compression with flow transfer to reduce bandwidth needs while preserving visual details in user videos.
Authors
Qizhe Wang, Jiaye Fu, Feng Ye, Haocheng Tang, Zetian Song, Chuanmin Jia, Siwei Ma, Jiaqi Zhang
Abstract
Recent advances in video generation have enabled generative video compression to reconstruct details lost during coding using pretrained spatiotemporal priors. However, a substantial gap remains between codec-induced degradation and the perturbation trajectories learned by flow-matching models. In this paper, we identify a compression-dependent directional mismatch that degradation-strength-based timestep calibration alone cannot resolve. Compressed latents remain strongly aligned across codec operating points, while their reconstruction residuals vary substantially in magnitude and direction. To bridge this gap, we propose RIFT, a Residual-Informed Flow Transfer framework for one-step generative video compression. At its core, RIFT introduces a Residual Geometry Inducer (RGI) that converts the pretrained flow prediction into a content-adaptive restoration update using decoder-visible codec cues. RGI integrates flow-timestep conditioning with a learned full-band residual-strength estimate and content-conditioned direction and gain modulation, aligning the orientation and magnitude of the update with the compression-dependent residual geometry. Extensive experiments on three public video compression benchmarks show that RIFT outperforms competing generative video compression methods. The code is publicly available on RIFT.