ODE-driven cross-attention improves image to point cloud matching accuracy
OCA: ODE-Driven Cross-Attention for Image-to-Point-Cloud Registration
Computer Vision and Pattern Recognition
Summary
Matching 2D images with 3D point clouds is hard because it’s tricky to figure out which parts correspond between them. The authors treat this matching process like a system of continuous equations, refining how features from images and 3D points pay attention to each other. This helps the computer learn better matches by reducing ambiguity. Their approach can be added to existing methods and improves the accuracy of matching in several tests.
What this means in practice
- •For 3d mapping engineers: Improve alignment of camera images to 3D point clouds for better spatial reconstruction using refined attention models.
- •For augmented reality developers: Enhance real-time integration of 2D camera views with 3D spatial data for more accurate AR device localization and object placement.
Authors
Pei An, Jiaqi Yang, Yulong Wang, Siwen Quan, Liangliang Nan
Abstract
Cross-attention is a crucial component in learning-based image-to-point-cloud (I2P) registration. Although existing cross-attention mechanisms have achieved promising progress, attention ambiguity remains a fundamental challenge that hinders the learning of discriminative 2D-3D correspondences. To address this problem, we revisit cross-attention and establish ordinary differential equations (ODEs) to model the ideal I2P feature interaction. Based on this formulation, we develop an ODE-driven cross-attention (OCA) module that refines feature representations and attention matrices through ODEs. In practice, OCA can be seamlessly integrated into existing I2P registration frameworks. To validate its effectiveness, we incorporate OCA into five state-of-the-art baselines and evaluate on four public benchmark datasets. Experimental results demonstrate that OCA improves registration recall by up to 5\%, 9\%, and 15\% under the standard, fine-tuning, and zero-shot settings, respectively.