Study reveals how people use AI helpers for cooking and DIY tasks
Large-Scale User Behavior Analysis in Multimodal AI-Assisted Manual Task Execution
Human-Computer InteractionArtificial IntelligenceComputation and Language
Summary
Many people use AI-powered assistants that combine voice, text, images, and videos to help with tasks like cooking or fixing things around the house. The authors analyzed thousands of real users to understand how they interact with these helpers and what they want to achieve. They discovered different ways users talk to the assistants and what makes people satisfied with the experience. Their study points out new ideas for improving how these AI helpers are designed and how to keep users engaged during tasks.
Conversational Task AssistantMultimodal interactionUser behavior analysisUser intentUser satisfactionDialogue systemsReal-world usageTask engagementInteraction design
Authors
Rafael Ferreira, Diogo Tavares, Diogo Glória-Silva, David Semedo, João Magalhães
Abstract
Conversational Task Assistants (CTAs) are multimodal dialogue systems that support users in complex real-world tasks such as cooking and DIY through voice, text, image, and video interactions. Prior user studies have focused on controlled settings, leaving limited understanding of real-world CTA usage at scale. In this work, we present a large-scale study of CTA usage based on thousands of users in-the-wild. Our large-scale real-world data analysis unveils new understandings of (i) user-CTA interaction flows, (ii) user intents, (iii) user conversational traits, and (iv) behavioral factors associated with user satisfaction. Our findings reveal key opportunities for future research in CTAs, particularly in user interaction design and task engagement, concluding with concrete design guidelines.