Phase based method improves video scene reconstruction from unsynced cameras
Phase-DGS: Phase-guided Dynamic Gaussian Splatting from Unsynchronized Multi-view Video
Computer Vision and Pattern RecognitionGraphics
Summary
Reconstructing 3D scenes from videos taken by multiple cameras usually needs all the cameras to be perfectly synced, which is hard and expensive to do. The authors propose a new way to match video frames from unsynced cameras by using the phase of the motion cycle instead of relying on exact timestamps. This means frames that show the same moment in motion can be matched even if the cameras recorded at different times. Their method improves video quality significantly under challenges like delays or missed frames and works with existing reconstruction systems.
What this means in practice
- •For film post-production teams: Combine footage from unsynchronized cameras to create accurate 3D reconstructions without costly hardware synchronization setups.
- •For augmented reality developers: Integrate multi-view videos with temporal misalignment into AR systems by matching frames via motion phase rather than timestamps.
Authors
Hosung Jeon, Jun Young Jeong, Sangwoon Kwak, Joonsoo Kim, Sangmin Kim, Jaesik Park, Won-Sik Cheong, Hyon-Gon Choo
Abstract
Spatiotemporal scene reconstruction with Dynamic Gaussian Splatting (DGS) fundamentally depends on perfect temporal synchronization across all cameras, an assumption rarely achievable in practice. Industry-standard synchronization requires expensive specialized hardware and complex setup, while data-driven alternatives including audio-based alignment, geometry matching, and pose tracking remain limited by restrictive environmental or scene-specific assumptions. To overcome these limitations, we propose Phase-DGS, a framework that replaces conventional timestamps with a semantic phase representing each frame's inherent state within the underlying motion cycle. Frames capturing the same motion state share the same phase regardless of which camera records them or when, naturally establishing spatiotemporal correspondence across unsynchronized cameras. We validate Phase-DGS under temporal offsets, frame drops, and camera freezes, achieving up to +8.5 dB PSNR improvement over baselines while enabling seamless integration with existing DGS backbones including RealTime-4DGS and FreeTimeGS.