Trust influences how people share tasks with multimodal ai systems
Following Delegation Traces to Study Human Oversight in Interaction with Multimodal AI
Human-Computer InteractionArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
People working with AI systems that understand both video and text have to decide how much to rely on the AI for help. This study found that how much users trust the AI shapes whether they reject, check, or fully delegate tasks to it. Interestingly, even when they fully trust the AI, many still prefer to verify its answers. The study also found people tend to delegate more as they gain experience, showing trust ratings alone don’t tell the full story. The authors suggest tracking how people actually hand over tasks to AI as a better way to study this collaboration.
What this means in practice
- •For product managers for ai tools: Design user interfaces that observe and adapt to how users delegate tasks to multimodal AI, improving trust and collaboration.
- •For customer support teams: Implement support workflows that factor in user tendencies to verify AI answers even when trusting the system, aiding oversight.
Authors
Agnese Chiatti, Matteo Matteucci, Viola Schiaffonati
Abstract
As multimodal AI systems are becoming our everyday thought partners, the shift towards studying how users transfer control to these systems becomes especially urgent. In this paper, we report an overview of findings from an online study with n=56 AI-literate participants and 392 Video Question Answering trials. After inspecting each multimodal AI response, participants rated task-specific trust and selected one of three intended policies for an analogous future task: reject AI assistance, use it with verification, or delegate fully to the AI. Within participants, post-interaction trust strongly predicted the user's delegation choices (OR=14.97, 95% CI [7.14,31.37]). Trust and delegation nevertheless remained distinct. Verification was the dominant policy and often co-occurred with high trust. Delegation also increased with task order after adjustment for momentary and participant-level trust (OR=1.22, 95% CI [1.02,1.45]), indicating an experience-related component that self-reported trust ratings alone could not capture. These findings motivate a shift from intended delegation to a non-intrusive observation of how delegation is enacted in collaborative human-AI tasks. We therefore conclude by proposing enacted delegation traces as a methodological framework for future studies of multimodal and agentic AI use.