Summary
Finding the location of a photo by comparing it to satellite images usually needs many matched examples and complex image models. The authors tested whether describing scenes in words could help match ground photos to satellite views without any training. They found that descriptions alone are not detailed enough to pick the exact location among many options, but they can help distinguish locations if the search is already narrowed down. Their approach also helps explain which parts of the descriptions agree or differ, offering more insight than traditional image matching. However, fine visual details that separate nearby spots are lost when converting images into words.
What this means in practice
- •For mapping service developers: Improve interpretability in location-matching tools by using language-based descriptions to explain which visual scene elements agree between ground and satellite views.
- •For robotics navigation teams: Use coarse geographic priors combined with language descriptions to better verify the location of a robot in urban areas with satellite and ground images.
Abstract
Cross-view geo-localization is commonly solved as an image retrieval problem, matching a ground-level image against a database of satellite tiles through a jointly trained embedding. Such models are accurate, but they need large paired supervision and cannot show what evidence supports a match. In this paper, we study a different question: how much of this task can be solved through language alone? We prompt a multimodal large language model (MLLM) to describe each ground panorama and each satellite tile as structured text, and localize by comparing these descriptions. No component is trained. We evaluate on 9,826 VIGOR pairs from four U.S. cities, in three settings. First, the descriptions are faithful but not discriminative. They agree closely across the two views, yet ranking the full pool by description similarity almost never returns the correct tile (0.39% Recall@1). Second, we narrow the pool to ten neighboring tiles, as a coarse prior would do. The same descriptions now become useful: an MLLM judge that scores structural consistency doubles random ranking and matches a strong lexical baseline. It also states which fields of the two descriptions agree and which conflict, which an embedding distance cannot do, and which we see as a step toward interpretable localization. Third, we place the judge on a trained visual retriever. On the queries it ranks wrongly, reranking from images works, while reranking from our descriptions does not (23.5% against 10.7% Recall@1). Scene structure survives the conversion into language, while the fine appearance detail needed to separate nearby places does not. Code and prompts are publicly available at https://github.com/AyeshAbuLehyeh/GeoLingual.