Gaze patterns use same motions differently across tasks and settings
The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences
Machine Learning
Summary
People look around using eye movements that can be broken down into small motion parts called motion tokens. The authors found that while these motion tokens are common, the way they are combined changes depending on what a person is looking at or doing. This means eye movement sequences are unique to each task or environment, making it hard to combine data from different sources without losing important details. The authors also offer a way to study these eye movements using ideas from information theory.
What this means in practice
- •For user interface designers: Design interface elements tailored to task-specific eye movement patterns revealed by local gaze sequence structures.
- •For human-computer interaction engineers: Improve gaze-driven interaction systems by adapting models to domain-specific gaze token transitions rather than pooling diverse datasets.
Authors
Virmarie Maquiling, Enkelejda Kasneci
Abstract
Human gaze is the primary high-bandwidth interface through which we parse the semantics of our environment. While often modeled as a sequence, the degree to which gaze exhibits transferable local structure across domains remains underexplored. Using motion tokenization as a diagnostic lens across four diverse datasets, we investigate the transferability of local sequence structure. We demonstrate that while gaze can share a reusable motion-token vocabulary, identical motion tokens are organized into different local transition patterns depending on the dataset and task context. Our results show the limitations of naive data-pooling under low-semantic motion-token representations and provide a framework for evaluating behavioral sequences through information-theoretic diagnostics.