Pair-relative partial equivariance improves rotation-robust keypoint matching
Pair-Relative Partial Equivariance for Rotation-Robust Keypoint Descriptor Matching
Computer Vision and Pattern RecognitionComputational Geometry
Summary
Matching keypoints between images is harder when the images are rotated a lot. Traditional methods assume a single fixed way that descriptors rotate, which can be too strict and fail in practice. The authors propose a more flexible approach called pair-relative partial equivariance that adapts to how keypoints actually rotate relative to each other in image pairs. Their method, PaSteer, improves matching robustness under large rotations without hurting performance on upright images.
What this means in practice
- •For computer vision engineers: Improve robustness of feature matching in image alignment and 3D reconstruction pipelines that involve arbitrary rotations.
- •For mobile robotics developers: Enhance visual localization and mapping algorithms with better rotation-robust keypoint matching in environments with varied orientations.
Authors
Jongmin Lee
Abstract
Learned keypoint descriptors trained on upright imagery often fail under large in-plane rotation. Steerers address this by applying a single exact SO(2) representation to the descriptor space, but this global group constraint can be too rigid for keypoint descriptors, whose angular responses need not follow a single group representation. We propose pair-relative partial equivariance, which measures equivariance under the empirical distribution of relative rotations in image pairs, and PaSteer, a relaxed descriptor-steering framework that allows the steering map to leave the exact representation manifold while retaining the lightweight, backbone-free design of steering. PaSteer instantiates this relaxation with two variants: an identity-preserving residual steerer (PaSteer-soft) and a free-form Fourier steerer (PaSteer-free). We show that the per-angle optimal linear steerer for a fixed non-equivariant descriptor generally does not obey the group composition law, explaining why exact representations can be suboptimal in the frozen-descriptor regime. On Roto-360, MegaDepth-1500, AIMS, and IMC-PT, PaSteer consistently improves rotation-robust matching over an exact steerer on frozen DeDoDe descriptors, especially under large relative rotations, with minimal upright degradation.