PolyDebate: A Game-Orchestrated Multimodal System for Debate Skills Practice and Evaluation
2026-08-17 • Human-Computer Interaction
Human-Computer InteractionComputation and Language
AI summaryⓘ
The authors created PolyDebate, a game-like system that helps people practice English debating by talking and using visual cues, not just writing. It pairs learners with an AI opponent and uses tools like skill cards and coins to make learning strategies clear and fun. The system listens to what the learner says and watches how they deliver it, then gives feedback using smart AI judges. The authors tested PolyDebate in four studies and found it effectively combines interactive debate, gamified learning, and detailed feedback to improve debate skills.
debateAI opponentmultimodal assessmentargumentationLLM judgesgamificationoral deliveryfeedback systemUnity 3Dpersuasive communication
Authors
Jianing Yin, Weng Pan Kuan, Xiaoyun Liu, Zhiyuan Wen, Yuxuan Li, Milos Stojmenovic, Jiannong Cao
Abstract
Debate is a structured form of persuasive communication that trains argument construction, rebuttal, oral delivery, and audience awareness. These skills are valued in education, language learning, and professional communication. Recent AI debate systems and LLM-based judges have advanced argument generation and debate evaluation, but most remain text-centered and rarely support learners through a complete multimodal practice experience. We introduce PolyDebate, a game-orchestrated multimodal system for English debate practice and evaluation. PolyDebate guides learners through staged one-on-one (1v1) debates with an AI opponent, while skill cards, props, and coins make persuasive strategies explicit and turn practice into a game-like interaction. During each session, the system captures learner speech and visual delivery evidence, generates context-aware opponent responses, and produces rubric-informed stage-level and overall feedback. PolyDebate is available as both an immersive Unity 3D game version and a web platform version that share the same workflow and evaluation services. Four studies covering AI opponent quality, evaluation coverage, AI judge feedback, and user perception show that PolyDebate brings debate interaction, gamified scaffolding, multimodal assessment, and structured feedback together in a practical workflow for debate skills practice. The demonstration video is available at https://youtu.be/mHwBG1_8Ebk.