Assessing Learning Processes with Multimodal Data in Virtual Reality Learning Environments
Human-Computer InteractionEmerging Technologies
Summary
The authors studied how people learn inside a virtual reality (VR) escape room by looking at more than just test scores before and after the activity. They collected different kinds of data like how players interacted with puzzles and what they said out loud while thinking through problems. By combining these data types, the authors aimed to better understand whether participants were truly reasoning or just guessing. Their work shows how VR can be used not just for learning content but also for analyzing learning processes in real time.
Authors
Eileen McGivney, Oluwatomilade Olarinde, Erica Kleinman, Kaylah Facey, Rana Jahani, Shripad Agashe, Manav Varma, Seth Cooper, Casper Harteveld
Abstract
Assessing learning in virtual reality (VR) environments typically relied on traditional pre-post content retention tests, revealing little about the process of learnng within such immersive environments. Multimodal data from player activity in VR is promising to better measure learning processes and higher-order skills, but little research in the learning sciences has explored how such data can be combined to provide meaningful measures. To address this, we explored multimodal sources of data from a VR escape room containing hands-on logic puzzles that require problem-solving strategies and engagement in verbal metacognitive reflections. We leverage VR's affordances to immerse users in sustained dialog and promote multimodal interactions in a digital environment to understand how logfile and verbal data reveal participants' reasoning skills versus guesswork.