FoundationVIO improves real-time visual odometry with inertial scale correction
FoundationVIO: Enforcing Metric Scale in Feed-forward Visual-Inertial Odometry
Robotics
Summary
Visual odometry systems use images and motion sensors to track movement and build 3D maps, but they often struggle to know the true physical size of what they see. The authors introduce FoundationVIO, which combines image-based predictions with inertial sensor data to correctly measure scale, making the movement tracking and maps more accurate. Their method runs fast enough for practical use on common laptops and works with regular cameras and motion sensors. This approach also adds a way to fix accumulated errors over time for better overall accuracy.
What this means in practice
- •For robotic system developers: Create robots that track their position and build accurate 3D maps indoors and outdoors using standard cameras and inertial sensors with real-time performance.
- •For augmented reality app builders: Develop AR applications that maintain correct scale and stable positioning by integrating visual and inertial data for better user experiences on mobile devices.
Authors
Jin Rhee, Yuhao Zhang, Frank Dellaert, Maurice Fallon
Abstract
Feed-forward geometric foundation models (GFMs) directly predict dense geometry and camera poses from batches of images. Emerging visual SLAM methods chain together consecutive GFM batches to form dense point cloud maps. A core challenge is how to address the ambiguous geometric scale of GFM predictions: independent predictions are locally consistent up-to-scale but not constrained by any direct physical measurements. Models trained on data with known scale improve the consistency of GFM reconstructions but again do not directly measure it. Consequently, trajectories and 3D reconstructions from GFM-enabled SLAM methods inherit this issue. Motivated by parallels with classical visual-inertial odometry, we introduce FoundationVIO which integrates inertial sensing with GFM predictions to recover metric scale reconstruction using a factor-graph based approach. We demonstrate odometry estimates competitive with classical state-of-the-art VIO methods and accurate dense reconstructions that outperform GFM-enabled SLAM methods. We also add a minimal GFM-based loop-closure mechanism to create a full Visual-Inertial SLAM system capable of creating globally accurate reconstructions and pose estimates. We evaluate our method on three datasets with well known sensor systems and demonstrate its compatibility on ubiquitous camera and IMU sensors. The method runs at approximately real-time on a consumer-grade laptop GPU. Code: https://github.com/ori-drs/FoundationVIO.