Gaze signals help track understanding in cooperative tasks
Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX
Computation and LanguageHuman-Computer Interaction
Summary
When people work together and share information unevenly, they need to make sure they understand each other. This paper looks at how where people look (their gaze) can show if they are on the same page. The authors studied two different conversation datasets and found that people focus more on the task and less on their partner when they understand each other well. The patterns of gaze help reveal this coordination, especially for the person explaining or leading the task.
What this means in practice
- •For human-computer interaction designers: Improve conversational agents by using gaze cues to better detect user understanding during task-focused dialogues.
- •For team collaboration software developers: Incorporate gaze analysis to support monitoring and enhancing shared understanding among remote collaborators in task-oriented settings.
Authors
Nan Li, Albert Gatt, Massimo Poesio
Abstract
In collaborative tasks with asymmetric information, participants coordinate their understanding through interaction. We ask whether gaze provides evidence about grounding across two such tasks. Working from discrete behavioral annotations, we map HCRC MapTask (Anderson et al., 1991) and MUNDEX (Türk et al., 2023) into a shared partner/task/away vocabulary and compute gaze features around task-relevant dialogue units. In both corpora, aligned reference interpretations (MapTask) and UND (understood) judgments (MUNDEX) are associated with more task-directed gaze and with less partner-directed gaze, lower gaze entropy, and fewer gaze transitions. The associations are clearest for the participant leading the task: in giver-produced references, and in explainer judgments, which also co-vary with the explainee's gaze. In same-speaker MapTask reference chains, the speaker's gaze entropy is lower at the mention where a previously non-aligned referent becomes aligned. The best gaze feature groups improve modestly over controls under grouped cross-validation: temporal features in MapTask and raw proportions in MUNDEX. Because effects are small and several weaken when recurring participants rather than dialogues are the unit of inference, we treat gaze as one contributing cue to grounding, to be interpreted alongside task and dialogue context.