LandmarkLens: Predicting and Presenting Effective Landmarks for Mixed-Reality Urban Exploration
2026-08-31 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors studied how people with good and poor sense of direction (SOD) pay attention to landmarks while navigating virtual Tokyo neighborhoods. They found that the two groups looked at and described different kinds of landmarks. Using this information, the authors created LandmarkLens, a mixed-reality tool that uses AI to highlight important landmarks for navigation. When people with poor SOD used LandmarkLens, they got better at recognizing places, showing that focusing on landmarks can help improve spatial learning.
sense of directioncognitive mapsspatial navigationlandmarksvirtual realitymixed realityvision-language modelspatial learningattentionscene recognition
Authors
Chu Li, Yotam Sechayk, Jared Hwang, Jon E. Froehlich, Takeo Igarashi
Abstract
People with a poor sense of direction (SOD) struggle to build cognitive maps for effective spatial navigation, and existing navigation tools prioritize efficiency over spatial learning. To understand how navigation strategies differ by ability, we conducted a landmark attention study with 20 participants (ten good SOD, ten poor SOD) who navigated across four Tokyo neighborhoods in virtual reality (VR). We found systematic group differences in both gaze behavior and the types of landmarks they verbally identify as effective. Based on these findings, we built LandmarkLens, a mixed-reality (MR) navigation system that uses a vision-language model (VLM) to identify and highlight navigation-relevant landmarks. A follow-up study with eight poor-SOD participants showed improved performance in scene recognition, suggesting that guided landmark attention can support landmark-level spatial knowledge acquisition for people with poor SOD, a first step toward broader spatial learning.