Bidirectional flow improves 3D scene reconstruction and video quality
Bi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinement
Computer Vision and Pattern Recognition
Summary
Reconstructing detailed 3D scenes from a small number of views is hard because multiple 3D shapes can look similar when rendered. The authors show that errors in 3D geometry can hide behind how the scene looks in images. They create a method where improved video sequences help fix the 3D shapes, and the better shapes help improve the videos, working together through motion information. This back-and-forth process leads to clearer, more consistent 3D scenes and videos even from limited viewpoints.
What this means in practice
- •For 3d graphics developers: Improve the accuracy and consistency of 3D scene models reconstructed from limited camera views using bidirectional flow techniques.
- •For video post-production teams: Enhance video frames with temporally consistent restoration guided by 3D geometry for better quality in visual effects workflows.
Authors
Yuetong Wang, Jinsheng Quan, Yi Yang, Yawei Luo
Abstract
Sparse-view 3D scene reconstruction with 3D Gaussian Splatting (3DGS) is inherently underconstrained. Plausible renderings can also coexist with erroneous Gaussian geometry, as errors in positions or depths may be concealed by opacity, scale, and appearance; we term this failure mode Geometry Cheating. Existing regularization methods constrain geometry but remain limited to observed views, while video-diffusion-based methods complete unseen views yet mainly use them as RGB pseudo-supervision, underusing motion and temporal priors and lacking explicit geometry supervision. We present Bi-FlowGS, which uses optical flow to bridge generative view completion and Gaussian geometry regularization. Our plug-and-play Video-to-Geometry Flow Distillation (V2G) distills temporal correspondence priors from restored videos into Gaussian geometry to alleviate Geometry Cheating. Conversely, Geometry-to-Video Flow-Guided Restoration (G2V) uses the current 3DGS geometry to guide temporally consistent video restoration, providing more reliable generative supervision. Together, V2G and G2V form an implicit bidirectional co-refinement process, enabling restored videos and the optimized 3DGS scene to iteratively improve each other. Experiments demonstrate improved rendering quality and geometric consistency across wide-baseline and unbounded 360° benchmarks.