4D dynamic scenes reconstructed from six sparse cameras using priors and AI
Sparse-View 4D Gaussian Splatting via Spatiotemporal Priors and Generative Assistance
Computer Vision and Pattern Recognition
Summary
Capturing detailed videos of moving scenes usually needs many cameras, but this paper focuses on doing it with only six widely spaced ones. The authors combine clever guesses about space and motion with AI-generated images to fill gaps that the few cameras miss. Their approach helps make clearer videos of dynamic scenes by improving how 3D shapes and movements are estimated over time. This new method performed best in a recent competition for sparse camera dynamic scene reconstruction.
What this means in practice
- •For volumetric video developers: Build dynamic scene reconstructions from only six cameras by integrating spatial and temporal priors with generative image guidance.
- •For virtual reality content creators: Generate high-quality 4D reconstructions of real-world dynamic environments using fewer cameras and AI-enhanced intermediate views.
Authors
Shengqi Wang, Zhengxian Yang, Kaiwen Tian, Yang Liu, Bowen Liu, Hua Du, Taicheng Huang, Jiamin Wu, Tao Yu
Abstract
We present a 4D Gaussian Splatting framework for the Sparse-View Track of the SIGGRAPH Asia 2026 Volumetric Video Challenge, which requires dynamic scene reconstruction from only six cameras with wide baselines. To achieve robust dynamic reconstruction under such sparse views, our framework integrates three components. (1) Region-adaptive spatial priors: We use foreground masks to guide Gaussian initialization and mask voting to control densification separately for the dynamic foreground and static background. Background geometry is regularized using monocular depth aligned to metric scale. (2) Motion-consistent temporal priors: We provide supervision at intermediate times through frame interpolation and constrain projected Gaussian motion with estimated optical flow. (3) Generative assistance: We place virtual cameras in the widest angular gaps and restore their rendered images using a diffusion-based model conditioned on camera poses. The restored images are iteratively incorporated into training as pseudo-supervision. On the validation set, our framework improves full-frame PSNR from 25.60 dB for the baseline to 29.75 dB. On the official test benchmark, it achieves 30.04 dB full-frame PSNR and 27.88 dB foreground PSNR, ranking first overall in the Sparse-View Track.