AI gameplay support agent struggles to engage puzzle game players
Oops, Not Now: PEARL, a RAG-Based Support Agent for Gameplay and What Players Want from AI Help
Human-Computer InteractionArtificial Intelligence
Summary
Providing helpful AI assistance for puzzle games is challenging because the AI must understand the game's state and explain moves clearly. The authors built PEARL, an AI helper that uses two kinds of information—game data and player questions—to give context-aware advice in a programming puzzle game. However, in user tests, many players found PEARL frustrating or unhelpful, often preferring existing visualization tools. The study reveals why these AI tools sometimes fail and suggests directions for improving how AI supports gameplay learning.
What this means in practice
- •For game developers: Integrate AI agents that combine board state data and semantic retrieval to offer players contextual gameplay explanations.
- •For interactive tutorial designers: Develop AI-driven support tools that tailor educational gameplay hints based on natural language questions and game progress.
Authors
Jiahong Li, Sai Siddartha Maram, Atieh Kashani, Ulia Zaman, Zhiyu Lin, Cameron Marano, Roger Azevedo, Jichen Zhu, Magy Seif El-Nasr
Abstract
AI-powered gameplay support agents hold promise for game-based learning, yet grounding generative models in structured game data remains an open challenge. We present PEARL (Parallel Education Agent for Reflection and Learning), a dual-component Retrieval-Augmented Generation (RAG) system that combines semantic knowledge retrieval with structural board-state matching to deliver contextualized scaffolding in Parallel, a puzzle game for learning parallel programming. PEARL operates on two input streams (natural language queries and board topology), retrieving both conceptual explanations of gameplay moves and peer-generated board states as evidence: capabilities unavailable to a standard Large Language Model (LLM) with game state access alone. In a qualitative evaluation (N=10) comparing PEARL against an existing community-based Open Player Model (OPM) visualization system, participants preferred the visualization system on perceived usefulness and reported higher frustration with PEARL; five of ten minimized or abandoned the AI tool during play. Proactive delivery, generic responses, and trust deficits drove disengagement, while a subset of four participants found PEARL's grounded explanations complementary to visualization in specific contexts where they initiated the interaction. We position PEARL as a deployed design probe whose failure modes inform a concrete design agenda for AI gameplay support, captured as seven open problems for the community.