Adaptive hands-free guidance helps blind users find objects indoors
Touvigation: Embodied Adaptive Object Acquisition for Blind and Low-Vision Users in Unfamiliar Indoor Environments
Human-Computer InteractionArtificial Intelligence
Summary
Blind and low-vision people often struggle to find things in new indoor places. The researchers created Touvigation, a system that gives real-time, body-focused directions to help users find and grab objects without needing to use their hands. It changes the type of guidance as users move from walking to reaching out and feeling for the item. In tests, Touvigation helped all participants find objects faster and with less mental effort than other tools or searching alone.
What this means in practice
- •For assistive technology developers: Build indoor navigation aids that provide real-time, body-relative directions to help blind users find objects more reliably and quickly.$Commercial implications: Enables new accessible products for blind users that outperform existing vision-language assistants in object acquisition.
- •For smart home system designers: Incorporate adaptive spatial guidance into devices to support visually impaired residents in locating household items in unfamiliar rooms.
Authors
George Xi Wang, Xiangyu Li, Shaoyue Wen, Jiaqian Hu, Junan Xie, Yupeng Wang, Ziyue Shi, Qijun Chen, Maaike Bouwmeester, Yuhua Jin, Jing Qian
Abstract
Blind and low-vision users often face challenges when locating and physically acquiring objects in unfamiliar indoor environments. Existing vision-language-model-based assistants can provide semantic descriptions but may introduce latency, hallucinations, and guidance that is poorly aligned with embodied action. We present Touvigation, a hands-free object acquisition system that combines vision-language understanding with persistent local spatial modeling to provide low-latency, body-relative guidance. Drawing on formative interviews with eight blind and low-vision participants, we design a multi-stage guidance framework that adapts spatial references as users transition from orienting, to walking, to reaching and tactile verification. We evaluated Touvigation with 12 blind and low-vision participants against a multimodal large-language-model assistant and unassisted search. Touvigation achieved 100% task success, compared with 58% for the multimodal assistant and 85% for unassisted search, while reducing completion time and cognitive workload. Our findings demonstrate how persistent spatial grounding and adaptive embodied guidance can improve object acquisition for blind and low-vision users.