Real-time video compression using deformable 2D Gaussian splatting
Deformable 2D Gaussian Splatting for Efficient 4K Video Compression
Computer Vision and Pattern Recognition
Summary
Ultra-high-definition videos require a lot of storage and are slow to decode in real time. The authors propose a new way to compress videos by using a 2D Gaussian Splatting method combined with a lightweight deformation network. This approach can compress groups of video frames quickly and efficiently, resulting in better video quality compared to the popular H.265 standard and other recent AI-based methods. Their method is especially suited for very high-resolution videos, like 4K.
What this means in practice
- •For video streaming providers: Implement faster, more efficient 4K video compression to reduce bandwidth use while maintaining visual quality.$Commercial implications: Enables streaming services to deliver high-resolution content at lower data costs with improved user experience.
- •For multimedia software developers: Build video encoding and decoding tools that leverage 2D Gaussian splatting for real-time high-definition video compression.
Authors
Chenhao Zhang, Fengqing Zhu
Abstract
Ultra-High-Definition (UHD) video presents significant challenges for efficient storage and real-time decoding. Learning-based methods, such as Neural Video Compression (NVC) and Implicit Neural Representations (INR), achieve competitive rate-distortion performance but suffer from high decoding latency and excessive memory usage. Meanwhile, Gaussian Splatting has recently attracted attention in the computer graphics community due to its ultra-fast rendering and high-fidelity visual quality. Despite these advantages, its application in video compression remains largely unexplored. To bridge this gap, we propose a real-time video compression framework that represents and compresses a Group of Pictures (GOP) using a coarse-to-fine multi-scale 2D Gaussian Splatting (2DGS) structure coupled with a lightweight deformation network. Experiments demonstrate that our method delivers rate-distortion performance in LPIPS that surpasses H.265 and other state-of-the-art learning-based video compression methods. Our work demonstrates the potential of Gaussian Splatting as a practical solution for efficient high-resolution video compression.