Contextual clues in physical spaces improve AI aided decisions
How Contextual Information in Physical Environments Shapes Decisions with AI
Human-Computer Interaction
Summary
It's hard to make good choices when we can't see everything and rely on AI that sometimes errs. The authors studied how knowing context, like how likely an object is present and how much of the scene is visible, influences decisions about whether a key is on a desk. They found that when the context suggested the key was probably absent and part of the scene was invisible, people made better decisions with AI help. However, this improvement mostly came from the different situations and AI responses people considered, not from changing their judgment styles. The study suggests future AI tools should offer evidence that helps confirm hidden facts.
What this means in practice
- •For augmented reality developers: Design AI-enabled AR interfaces that consider environmental context to improve user decisions about partially visible objects.
- •For safety-critical operators: Use contextual AI advice where object presence is uncertain to reduce errors in supervised monitoring tasks.
Authors
Omar Elgohary, Amanda Elvarsdottir, Vannessa Lee, Zhu-Tian Chen
Abstract
We studied whether contextual information helps people make better decisions with fallible AI in a partially visible environment. In an AR study (N = 32), participants judged whether a rendered key was present on a physical desk, with or without AI advice. We varied target prevalence and scene visibility. Focal analyses excluded visible targets and AI suggestions that could be directly checked. Within this subset, the combined context of a stronger prior toward absence (lower prevalence) and a target-free half increased AI-assisted accuracy by 9.5 percentage points. An exploratory analysis accounted for most of this increase through changes in the cases and AI suggestions being evaluated; changed participant responses did not significantly improve accuracy beyond maintaining baseline response patterns. The stronger prior reduced agreement with Present suggestions, while the target-free half lengthened decision time. We did not detect more selective agreement with correct versus incorrect advice. These findings motivate evaluating situated assistance on matched cases and designing ways to obtain evidence that can resolve claims about hidden states.