Boosting Neural Video Codec via Scale-Driven Online Flow Refinement
2026-06-22 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address a problem where advanced neural video compression methods struggle to handle new and complex motion patterns they weren't trained on. They introduce a new approach called SOFR that improves motion estimation by combining information from different motion scales without needing extra training. SOFR adjusts dynamically based on how well it predicts motion and uses a smart method to keep the process reliable and efficient. Their tests show it effectively reduces data size needed for videos while adding almost no extra processing time.
Neural Video CodecMotion EstimationOnline RefinementScale-Driven FusionBitrateWarping ErrorPSNRMS-SSIMDomain GeneralizationVideo Compression
Authors
Tiange Zhang, Rongqun Lin, Haocheng Tang, Xiandong Meng, Weijia Jiang, Zhimeng Huang, Siwei Ma
Abstract
Although state-of-the-art neural video codecs (NVCs) have achieved remarkable performance, they suffer from limited generalization when encountering complex motion patterns unseen during training. To bridge this domain gap without the expensive cost of online fine-tuning, we propose a Training-Free Scale-Driven Online Flow Refinement (SOFR) method. Serving as a plug-and-play module, SOFR integrates motion information from coarse and fine scales and dynamically fuses them according to warping accuracy, effectively rectifying motion estimation errors with negligible computational overhead. Furthermore, we design a rate-aware strategy that selects different dynamic fusion strategies according to bitrate modes, and employs a reliability check based on warping error to ensure robustness. Extensive experiments on the USTC-TD dataset verify the effectiveness and generalization of SOFR across various NVC frameworks, including DCVC-SDD, DCVC-FM, and EHVC. Notably, it brings an average of 2.84% and 4.05% bitrate savings in terms of PSNR and MS-SSIM, respectively, to DCVC-FM with negligible coding time increase. Our code is available at https://github.com/SunnyMass/SOFR.