Robust compression method reduces data for dynamic 3d gaussian splats
Robust, Estimator-Agnostic Dynamic 3DGS Compression
Computer Vision and Pattern Recognition
Summary
Representing moving 3D scenes with Gaussian splats usually requires lots of data. The authors found a way to pack information from multiple frames into one set by adding a frame label to each Gaussian, turning a time problem into a space problem. This new method works well with many existing compression techniques and doesn’t rely on tracking the Gaussians over time or knowing how they were made. Their approach reduces data size significantly when tracking is present, and stays competitive even when tracking is absent.
What this means in practice
- •For 3d graphics developers: Compress dynamic 3D scenes more efficiently without needing motion estimation by using augmented Gaussian sets for storage and streaming.
- •For video coding engineers: Incorporate the technique to improve bitrate savings when encoding sequences of 3D point-based frames, especially for codecs supporting static 3D Gaussian splats.
Authors
Chenjunjie Wang, Zixi Huang, Yao Wang, Jona Ballé
Abstract
Dynamic 3D Gaussian splats (3DGS) model time-varying scenes using a separate Gaussian set per frame. While neighboring video frames are highly correlated due to smooth motion, Gaussian representations retain this correlation to varying degrees, depending on whether the estimator tracks them across time. Some 3DGS compression methods integrate the estimation to exploit temporal redundancy; here, we focus on robust compression regardless of the estimator. We concatenate groups of frames into one Gaussian set, augment each Gaussian with a frame index, and pass it to a static (i.e., non-temporal) 3DGS codec, converting temporal redundancy into spatial redundancy. Concatenated sets are spatially partitioned to limit memory. Our technique requires neither a motion model nor knowledge of the training method. Averaged over six N3DV sequences, all six static codecs achieve gains on tracked sets (-42.0% to -71.8% BD-rate) over per-frame coding. On untracked sets, all codecs except HGSC, which appears incompatible with our technique, remain competitive with per-frame coding (-3.5% to +5.0%). We further replace D-FCGS's I-frame coding with our technique while retaining its P-frame coding, yielding an overall BD-rate of -46.2%. We propose to visualize "trackedness" using an inter-frame similarity metric. The project is available at https://wcjj1236.github.io/d3dgs-benchmark.