Camera pose helps improve robot depth estimation from video
RIDE: Relocalization-Informed Depth Estimation with 3D Gaussian Splatting
Robotics
Summary
Estimating how far away things are in a video is important for robots but can be tricky. The authors created a method called RIDE that uses the robot’s known camera position to help measure distances more accurately in videos. It combines information about where the camera is with clues from a trained model to guess depth, even when data is incomplete or noisy. This makes the robot’s depth perception more precise and consistent over time.
What this means in practice
- •For robotics engineers: Improve robot depth sensing using camera pose data combined with pretrained video depth models for better navigation.
- •For augmented reality developers: Enhance dense depth maps in AR systems by integrating camera relocalization with video depth estimation for real-world scale accuracy.
Authors
Jiarong Lian, Zhe Xiao, Zhaoyang Zhang, Wei Li, Ruizhi Chen
Abstract
Render--match--PnP relocalization establishes correspondences between query image pixels and 3D map points for camera pose recovery, but their potential to support dense depth estimation is often overlooked. To exploit this geometric information, we present RIDE, which estimates dense metric depth from a robot's RGB stream. Given a metrically scaled 3D Gaussian Splatting (3DGS) model, RIDE combines sparse metric depth observations derived from PnP-RANSAC inlier correspondences with the geometric prior of a pretrained video-depth model. To handle uneven and intermittent observations, it integrates global and local depth correction with temporal memory, supporting depth estimation through short observation gaps after metric scale initialization. Trained on public RGB-D videos, RIDE is evaluated on robot sequences without fine tuning. Experiments show improved depth accuracy and temporal consistency over scale-only calibration, demonstrating how localization geometry can support both pose recovery and dense robot perception.