Children check AI suggestions differently based on agent style
"Do You Trust It... or Check It?": Exploring Children's Verification Behaviors in Game-Based Human-AI Collaboration
Human-Computer Interaction
Summary
Children interact with AI helpers when playing a biology game, but they don’t always know if AI answers are right or wrong. The researchers studied how kids check or question AI advice depending on how the AI behaves—as a partner, a tool, or a mix of both. Kids tended to ask questions more when the AI acted like a partner, and looked for evidence more when it acted like a tool. The study shows ways to design AI that encourages kids to think carefully and check facts when working with AI.
What this means in practice
- •For educational game designers: Design AI agents in educational games that promote children’s critical questioning and evidence checking during learning tasks.
- •For ai interface developers: Create AI assistants with interaction styles tailored to encourage user verification behaviors suited for different age groups.
Authors
Shuning Liu, Yuxin Song, Hanbing Li, Ziyue Wang, Mingying Lan, Fei Lv
Abstract
Children are increasingly interacting with AI agents in learning and problem-solving contexts, where AI suggestions may appear fluent, helpful, and socially engaging. However, children may not always be able to judge whether such suggestions are accurate, incomplete, or misleading. This paper presents a game-based human-AI collaboration probe designed to examine how children verify AI suggestions during task-based interaction. The system embeds AI suggestions within a biological knowledge reasoning game and compares three Agent interaction styles: partner-like, tool-like, and dual-state Agent. Instead of treating trust as a static post-task attitude, this study focuses on children's situated verification behaviors, including acceptance, evidence checking, questioning, revision, and continued AI use. A preliminary evaluation with children aged 8-10 combined interaction logs, post-task ratings, and semi-structured interviews. The findings suggest that Agent style shaped children's engagement and response pathways: partner-like interaction tended to support direct questioning, while tool-like interaction was more associated with evidence seeking. Children generally viewed AI as useful but fallible, and their questioning behaviors varied in reasoning depth. The study contributes design insights for supporting children's critical, reflective, and evidence-based use of AI in collaborative tasks.