DecoGS speeds up 3D video updates with less flicker and drift
DecoGS: Adaptive Static-Dynamic Decoupling of 3D Gaussians for Free-Viewpoint Video Streaming
Computer Vision and Pattern Recognition
Summary
Streaming 3D videos smoothly is hard because updating every part all the time causes flickering and slow performance. The authors propose DecoGS, a system that only updates parts of the video that change, ignoring static areas to avoid unnecessary work. This selective updating reduces flickering and keeps the video looking stable while running much faster. DecoGS also manages memory efficiently and achieves higher video quality than previous methods without needing large pretraining.
What this means in practice
- •For augmented reality developers: Improve real-time 3D scene updates in AR apps by reducing flicker and speeding rendering without pretraining on large datasets.
- •For video streaming engineers: Enable efficient delivery of live 3D videos by selectively updating dynamic regions, lowering bandwidth and improving viewer experience.
Authors
Idil Sulo, Alexey Supikov, Ilke Demir, Sainan Liu
Abstract
Streaming 3D reconstruction demands both speed and temporal fidelity, goals that existing methods undermine by updating every Gaussian every frame, even in static regions. We present DecoGS, a method for efficient online training of 3D Gaussians from streaming videos. Unlike prior methods that update the entire scene indiscriminately, DecoGS introduces an adaptive mechanism that selectively focuses optimization on spatiotemporal regions exhibiting motion or photometric changes. This targeted training strategy eliminates redundant updates that cause flickering and drift in nominally static regions, while enabling fast, high-fidelity scene updates. The pipeline further integrates region-aware Gaussian management through gradient gating and efficient visibility filtering to maintain temporal coherence and a compact memory footprint. On N3DV and MeetRoom, DecoGS achieves 34.55 and 31.60 dB PSNR respectively, outperforming all streaming and offline baselines, while rendering at 261 FPS with $70\times$ lower temporal flicker than the best prior method, requiring no large-scale pretraining.