Duet4d improves 4d scene reconstructions from few camera views
DUET4D: Self-Corrective Coupled Guidance for Sparse-View 4D Reconstruction
Computer Vision and Pattern RecognitionComputational Geometry
Summary
Reconstructing 3D scenes over time from very few cameras is difficult and usually produces shaky or incomplete results. The authors introduce DUET4D, a method that connects a 4D model with generative video predictions in a way that lets them correct each other repeatedly. This self-correcting approach helps produce clearer and more accurate animated 3D scenes without adding strange artifacts. Compared to earlier methods, DUET4D achieves better quality and faster rendering.
What this means in practice
- •For graphics engineers: Create smoother and more accurate dynamic 3D scenes from limited camera setups for visual effects and virtual production.
- •For augmented reality developers: Improve real-time 4D scene reconstructions with fewer cameras to enable more immersive and consistent AR experiences.
Authors
Beizhen Zhao, Nanjie Yao, Boyi Fu, Qingsong Yan, Haofan Ren, Kaiyong Zhao, Hao Wang
Abstract
Sparse-view 4D reconstruction is highly under-constrained when only a few cameras are available. Generative priors can recover missing appearances, but video models are computationally impractical for iterative reconstruction, whereas single-view models produce temporally inconsistent pseudo-observations. Moreover, directly absorbing uncertain generative outputs into Gaussian optimization often creates redundant primitives and floating artifacts. To address these challenges, we propose DUET4D, a self-corrective coupled guidance framework for sparse-view 4D Gaussian splatting. The core of our approach is a self-corrective framework where the 4D representation guides generative prior for spatio-temporal consistent supervisions. We first construct sequential alignments from current Gaussian evidence to steer the latent restoration. Then we use Sequential Fidelity Resampling to lift generative supervisions into Gaussian space to selectively distinguish repeatedly supported primitives. The corrected representation then provides more reliable Gaussian evidence for the next restoration cycle. Extensive experiments demonstrate that DUET4D significantly improves rendering quality and rendering efficiency.