Bayesian updates improve 3D reconstruction from long video streams

BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates

Computer Vision and Pattern RecognitionArtificial Intelligence

Summary

Building 3D images from videos can be tricky because errors add up over time, making the model worse. The authors introduce a method called BAT3R that uses smart checking and updating techniques based on probability math to keep the 3D model accurate as new video frames come in. Their approach protects well-learned parts of the scene while updating only what is new or uncertain. Tests on many standard datasets show BAT3R works better than existing tools, especially for long or complex video sequences.

What this means in practice

  • For robot navigation teams: Produce more accurate real-time 3D maps from continuous video input to support navigation in complex and changing environments.
  • For augmented reality developers: Maintain reliable 3D scene models over long sessions to improve user experience in AR applications using live camera feeds.

Authors

Lijian Li, Yuanpeng He, Yudian Zheng, Chung-ju Huang, Linyu Li, Dongming Jin, Wenpin Jiao, Zhi Jin, Chi-Man Pun

Abstract

Feed-forward transformer models have driven remarkable progress in 3D vision,yet their quadratic complexity renders them impractical for long video sequences. Streaming 3D reconstruction addresses this bottleneck by processing frames sequen?tially with constant memory. However, existing recurrent architectures inevitably suffer from progressive degradation over long sequences, as they fail to account for the intrinsic reliability of observations and the accumulated confidence of the historical state, leading to severe error accumulation. To address these issues, we introduce BAT3R, a training-free Bayesian adaptation framework comprising two synergistic modules: State Innovation Gating (SIG) and Bayesian-guided Adaptive State Updating (BASU). SIG acts as a distributional regularizer that anchors state evolution to its geometric initialization, effectively arresting representational drift during long-term inference. Building on this, BASU reformulates state update as a recursive Bayesian inference process within a Kalman filtering framework. By jointly modeling the temporal reliability of observations via information-theoretic gain and spatial relevance via cross-attention maps, BASU achieves an asymmetric spatio-temporal state fusion: well-consolidated historical regions are protected from redundant updates, while genuinely novel scene content is aggressively incor?porated. Extensive experiments on standard benchmarks (7-Scenes, TUM, ScanNet, KITTI, etc.) demonstrate that BAT3R consistently outperforms state-of-the-art online 3D reconstruction methods, with the most significant gains observed on long sequences and complex scenes.