Spatial layout improves screen reading for blind mobile users
Spatial Organization Is Accessibility Structure: Hierarchical Screen Reading of Mobile UIs
Human-Computer Interaction
Summary
Blind and low-vision users typically hear mobile screens read out as a long list, which makes it hard to understand how things are grouped or what they do. The authors studied how current screen readers fall short and created a new system called Spatial Screen Reader (SeeR) that uses the way items are arranged on the screen to organize information better. SeeR groups related controls, summarizes their functions, and lets users navigate these groups hierarchically. Tests showed it reduces unnecessary steps and helps users understand controls faster, especially on new apps, though it adds some effort when users know exactly what they want.
What this means in practice
- •For mobile app developers: Design apps that expose spatial groupings explicitly to improve accessibility for blind users navigating via screen readers.
- •For accessibility software engineers: Integrate spatial layout data into screen readers to create hierarchical navigation and reduce user effort for blind and low-vision users.
Authors
Yongxiang Hu, Xuan Wang, Qingyang Qiu, Yingjie Liu, Chun Kang, Ruiying Zeng, Xin Wang, Yangfan Zhou
Abstract
Screen readers for blind and low-vision (BLV) users flatten mobile UIs into a linear sequence. Modern mobile UIs, however, contain dozens of elements grouped by spatial layout. Such linear access is ill-suited for BLV users, who must infer each element's meaning and which elements form a functional unit. Drawing on a quantitative TalkBack audit and formative interviews, we present Spatial Screen Reader (SeeR), which builds a function-aware hierarchy from accessibility metadata and provides unit summaries with hierarchical navigation. Compared with TalkBack, SeeR reduces non-informative focus stops by 82.1% and navigation steps by 52.7%. With ten BLV participants, summaries helped them connect controls to their functions and skip nonessential elements, especially on unfamiliar pages, though extra navigation levels added effort when the target was already known. These findings recast spatial organization as accessibility information: apps and platforms should expose it, rather than leaving screen readers to infer it post hoc.