Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration
Computer Vision and Pattern Recognition
Summary
The authors address the problem of matching 2D images to 3D point clouds, which is important for tasks like robot navigation or augmented reality but is often disrupted by noisy data. They notice that existing methods struggle when the input data has many incorrect matches or noisy depth information. To fix this, they propose a new approach called Ex-Sim(3)-Reg that better handles noisy depth by reformulating the problem to account for errors explicitly. Their tests on several datasets show this method improves accuracy by up to about 25% compared to previous techniques. They also provide a theoretical explanation for why their method works.
Authors
Pei An, Muyao Peng, Junfeng Ding, Jiaqi Yang, Liangliang Nan
Abstract
Learning-based image-to-point-cloud (I2P) registration has garnered increasing attention in recent years. Nevertheless, existing methods still struggle with severe outliers under challenging scenarios with unseen, low-inlier, or distorted cases. A fast and robust 2D-3D correspondence pruning method is therefore highly desirable. Recently, a promising scheme lifts 2D-3D correspondences to 3D-3D correspondences using depth priors, casting correspondence pruning as a Sim(3) registration problem. However, depth priors estimated from monocular images are inherently noisy, which undermines the reliability of this scheme. In this paper, to explicitly model non-negligible depth noise, we reformulate correspondence pruning as an extended Sim(3) registration problem and propose a simple yet effective pruning algorithm termed Ex-Sim(3)-Reg. We further provide a theoretical analysis to justify the effectiveness of our method. Extensive experiments on the 7-Scenes, RGBD-V2, ScanNet, and TUM datasets demonstrate that Ex-Sim(3)-Reg achieves up to \textbf{24.7\% improvement} in registration recall over state-of-the-art baseline methods. Code is released at github.com/anpei96/ex-sim3-demo