Global method improves 3d reconstruction from long video sequences
GRF-Recon: Global Ray-Field Optimization for Long-Sequence Feed-forward Reconstruction
Computer Vision and Pattern Recognition
Summary
Reconstructing 3D scenes from long sequences of images is tricky because computers often run out of memory, get confused about details, and lose track of where the camera is over time. The authors introduce a new approach that looks at the whole video at once, rather than in small parts, to keep the 3D model stable and accurate. They use clever ways to include depth hints and connect different frames strongly to clean up noise and fix errors. Their system works well compared to existing navigation tools and keeps the 3D scenes consistent over large areas.
What this means in practice
- •For robotics developers: Create more accurate and stable 3D maps from long monocular camera paths to improve navigation and interaction in complex environments.
- •For augmented reality developers: Build globally consistent large-scale 3D scene reconstructions from camera video for immersive AR experiences with minimal drift and distortion.
Authors
Enpeng Li, Yunzhou Zhang, Zhiyao Zhang, Dexuan Lyu, Chenyu Wang, Chiyuan Cui, Cheng Cheng
Abstract
Feed-forward 3D reconstruction provides an efficient paradigm for scene modeling from image sequences. Scaling these models to large monocular scenarios are constrained by excessive GPU memory footprint, degraded local geometry, and long-term trajectory drift. Existing chunk-based optimization strategies provide limited geometric constraints and fail to maintain global consistency over extended trajectories. We present a unified framework for stable and scalable feed-forward 3D reconstruction from long monocular sequences. Our approach builds on coarse-to-fine trajectory alignment augmented by lightweight geometric prior injection. Distilling monocular geometric cues into the feed-forward backbone via LoRA adaptation improves depth accuracy on fine structures while preserving inference efficiency. We introduce a hybrid-weight sparse ray-field optimization that leverages high-frequency geometric features to guide local point-cloud refinement and enforce consistent inter-frame ray constraints. Unlike prior chunk-based methods, this establishes strong cross-frame geometric coupling while maintaining scalability. Finally, an efficient trajectory stitching strategy with joint ray-error optimization explicitly reduces accumulated drift. Extensive experiments show that our approach achieves competitive trajectory accuracy compared with representative SLAM systems, while maintaining globally consistent 3D reconstruction in large-scale scenarios.