Memory efficient 3d reconstruction improves camera pose stability
Think Locally, Refine Globally for Memory-Efficient 3D Reconstruction
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Reconstructing 3D scenes from long video sequences can use a lot of memory and sometimes cause errors in tracking camera positions. The authors propose a method called LoG-VGGT that processes short time windows locally while sharing important information between windows, reducing memory use. They add a module that improves the estimation of camera positions over the entire sequence to avoid drift. Their approach achieves better depth accuracy and stable camera poses on long sequences without needing extremely large memory.
What this means in practice
- •For augmented reality developers: Create real-time 3D maps from long video captures with improved accuracy and lower memory needs on mobile devices.
- •For drone operators: Maintain reliable 3D environment models from extended flight sequences for navigation and mapping despite limited onboard memory.
Authors
Jingke Zhou, Chenhang Ma, Zhizhou Zhong, Mingkai Liu, Zhuang Zhou, Yicheng ji, Binghua Su, Bo Cai, Xianliang Huang
Abstract
We propose LoG-VGGT, a memory-efficient framework for long-sequence 3D reconstruction that balances local temporal modeling with global camera consistency. Instead of relying on full global attention, our method introduces cross-window attention at a small subset of transformer blocks, enabling effective information propagation across adjacent temporal windows while keeping memory usage bounded. To mitigate long-term pose drift, we further design a global camera consistency refinement module, where camera tokens interact with compact register tokens via cross-attention to enforce scene-level constraints across the entire sequence. This design enables joint optimization of camera representations and significantly improves long-horizon pose stability without incurring the high cost of sequence-wide attention. Extensive experiments demonstrate that LoG-VGGT achieves improved depth accuracy and robust camera pose estimation across multiple long-sequence benchmarks, while delivering competitive streaming reconstruction performance.