Spherically consistent matching improves 360 degree image feature alignment
SCCM: Spherically Consistent Coarse Matching for ERP Dense Feature Correspondence
Computer Vision and Pattern Recognition
Summary
Matching features accurately across 360-degree images is tricky because the way these images are shown distorts distances and shapes. The authors improved this process by fixing these distortions right where the matching begins, using new methods that understand the spherical nature of the images. Their approach, called SCCM, works better than previous methods on standard 360-degree image datasets and even transfers well to new types of data without retraining. This helps computers better recognize corresponding points in panoramic images.
What this means in practice
- •For augmented reality developers: Improve pose estimation and view synthesis in 360 degree AR experiences by more accurately matching features across panoramic images.
- •For robotics navigation teams: Enhance omnidirectional SLAM systems by providing more reliable dense feature correspondences on spherical images for robot localization and mapping.
Authors
Gyeonggwan Lee, Eunsoo Im, Seunghwan Hong, Junghun Suh
Abstract
Equirectangular projection (ERP) is the standard representation for 360$^\circ$ imagery, and robust dense feature matching on ERP underpins panoramic stereo, view synthesis, and omnidirectional SLAM. Dense matchers trained on flat images degrade systematically on ERP because the chart introduces three coupled distortions -- topological, metric, and area -- that standard coarse matching and visibility estimation do not explicitly model. We show that correcting the three distortions at the coarse-stage interfaces where they arise -- pairwise distortions in attention, per-pixel distortion in covisibility gating -- improves PCK@$1^\circ$ from 0.229 to 0.275 on Matterport3D under a fixed coarse scaffold, with the refiner architecture unchanged -- our central result. Concretely, SCCM (Spherically Consistent Coarse Matching) augments a chart-naive cross-attention/dual-softmax coarse matcher with two sphere-derived priors: Spherical Positional Attention (SPA) pairs a yaw-periodic RoPE (topology) with a tangent-plane bias (metric), and Area-Aware Covisibility (AAC) applies a pre-sigmoid log-area correction (area). The chart-naive scaffold serves as a controlled reference, separating the scaffold-replacement effect from the spherical-prior effect. Instantiated in the RoMa V1 framework with the same frozen encoder, refiner architecture, and loss, SCCM also outperforms the ERP-native EDM (0.163) and an ERP-retrained RoMa V1 (0.198) under a unified ERP dense matching protocol, while perspective-trained matchers largely fail on ERP. It further transfers zero-shot to Stanford2D3D and, when trained on outdoor Holo360D, leads there as well.