Geometry semantic constraints improve ground camera localization with satellite images

How to Reduce Localization Ambiguity? Geometry-Semantic Constrained BEV Representation Learning for Satellite-Ground Localization

Computer Vision and Pattern Recognition

Summary

Satellite-ground localization finds where a ground camera is positioned within a satellite image, including which way it is facing. The problem is that features can appear similar or be placed at uncertain depths, causing confusion about exactly where the camera is. The authors propose a method called GeoSem-BEV that uses geometric rules about depth and height, plus semantic understanding of the scene, to better place features and tell them apart. This approach helps reduce errors in determining the camera’s location and orientation when tested on multiple datasets.

What this means in practice

  • For autonomous vehicle engineers: Improve vehicle positioning systems by better matching ground camera views to satellite maps despite ambiguous feature appearances.
  • For drone navigation teams: Enhance drone localization accuracy by integrating semantic and geometric constraints in matching onboard camera views to satellite imagery.

Authors

Junming Feng, Panwang Xia, Qiong Wu, Xudong Lu, Zeyu Jiao, Kun Lv, Zherong Wu, Yi Wan, Peifeng Ma, Li-Ta Hsu, Zhi Zheng

Abstract

Satellite-ground localization estimates the planar position and yaw orientation of a ground camera within a geo-referenced satellite image. Most recent methods map ground and satellite features into a shared bird's-eye-view (BEV) space and establish spatial correspondences. However, insufficient depth constraints can assign one ground feature to different distances along a viewing direction, creating geometric ambiguity in BEV feature placement. Similar appearances at different locations can also create descriptor matching ambiguity, while existing descriptor learning lacks explicit semantic supervision to distinguish them. We propose GeoSem-BEV, a geometry-semantic constrained BEV representation learning method. Radial depth supervision constrains distance assignment, and vertical height supervision constrains height aggregation. Shared explicit semantic supervision promotes consistent semantic predictions across views and helps distinguish locations with similar semantics. These constraints improve feature placement and descriptor discriminability, enhancing state-of-the-art BEV localization models. On VIGOR with unknown orientation, GeoSem-BEV reduces mean orientation error by 37.2% and 38.1% in the cross-area and same-area settings, respectively. The corresponding errors are reduced by 10.8% and 15.6% on DReSS-D. On KITTI-CVL, it reduces same-area mean orientation error by 26.8% under 10 degree orientation noise.