Pow3R-SLAM improves real-time 3D mapping with depth priors
Pow3R-SLAM: Real-Time RGB-D SLAM with 3D Reconstruction Priors
RoboticsComputer Vision and Pattern Recognition
Summary
When robots or devices move around, they need to know where they are and what the space looks like. Most cameras that help do this get confused when depth information is missing or sparse. The researchers created Pow3R-SLAM, which uses both color images and depth data better by filling in missing depth parts using smart prediction. This method makes the device's understanding of the environment faster, more accurate, and produces more detailed 3D maps compared to older methods.
What this means in practice
- •For robotic navigation teams: Enhance autonomous robot movement by generating more accurate and denser 3D maps from real-time RGB-D sensor data, even with sparse depth inputs.
- •For augmented reality developers: Create more precise and faster environmental mapping to improve AR device localization and interaction through smarter use of RGB-D sensors.
Authors
Christopher Kolios, Ishaan Mehta, Sasa Janjic, Yeganeh Bahoo, Sajad Saeedi
Abstract
We present Pow3R-SLAM, a real-time RGB-D simultaneous localization and mapping (SLAM) system that uses Pow3R for tracking and mapping. Inspired by MASt3R-SLAM, a recent work on monocular SLAM using two-view 3D reconstruction priors, we extend the work to incorporate depth as a prior on the network's prediction, rather than as geometry to fuse. Where traditional RGB-D SLAM systems struggle with sparsity in the depth images, Pow3R utilizes the available depth to give a better-conditioned pointmap, while inferring the depths in empty regions from the two-view photometric, depth, and intrinsic data. Evaluated against MASt3R-SLAM following its protocol on 24 sequences from TUM, 7-Scenes, and Replica, Pow3R-SLAM runs 1.6x faster in wall time, has 15% lower mean trajectory error, a 3.1x lower unscaled error, and produces denser maps, with a 30% lower Chamfer distance. We also introduce a hybrid variant that runs 2.1x faster than MASt3R-SLAM at 25.3 frames per second (FPS), while maintaining improved tracking and mapping accuracy. Against ORB-SLAM3 in RGB-D mode, Pow3R-SLAM is more accurate on TUM, 7-Scenes, and ETH3D-SLAM, and completes every TUM sequence. While Pow3R-SLAM can struggle on a small set of self-similar scenes, its overall performance shows that adding depth as a prior for two-view 3D reconstruction SLAM can be beneficial. A project webpage is available at: https://ChrisKolios.github.io/Pow3R-SLAM , and code will be made open-source upon acceptance.