Head-worn device measures learner engagement using brain and eye signals
E3Sense: Head-Confined Multimodal Sensing of Learner Engagement
Human-Computer Interaction
Summary
People learn better when teaching adjusts to how engaged they are, but measuring engagement can be tricky. The authors created a single wearable device that sits on the head and combines brain activity, eye movements, and skin signals to estimate how engaged someone is while watching educational videos. Their device could predict engagement levels better than just guessing the most common response. They also found that understanding how each person defines engagement helps improve predictions.
What this means in practice
- •For educational technology developers: Create adaptive learning platforms that adjust pacing by sensing learner engagement from a single wearable device combining brain, eye, and skin signals.$Commercial implications: Enables development of personalized teaching tools and engagement-responsive learning products using a compact head-worn sensor system.
- •For workplace training providers: Improve remote training effectiveness by measuring trainees' engagement levels through a convenient head-worn sensor during video-based instruction.
Authors
Sidharth Anupkrishnan, Itir Sayar, Jeongah Lee, Sri Harsha Musunuri, Guan-Ming Su, Madeline Endres, Ravi Karkar, Phuc Nguyen
Abstract
Engagement-aware learning systems could provide hints or adjust pacing when learners struggle. Prior engagement sensing work distributes sensors across or outside the body rather than consolidating them at one site, or reduces engagement to shared affect or a single dimension (from behavioral, emotional, and cognitive engagement). We introduce E3Sense, a head-worn platform that co-locates electroencephalography, eye tracking, and electrodermal activity to personalize engagement measurement. During a lab study we collected 450 ratings of engagement levels on a five-level ordinal scale while participants watched educational videos. For fifteen held-out participants, E3Sense achieved a within-one-level prediction score of 75.0%, compared with 63.0% for always predicting the most common rating. In an exploratory analysis of 18 participants from the same study who defined engagement, conditioning on learners' definitions raised the same measure by 6.9 points, from 64.6% to 71.5%. Our work provides a proof-of-concept of a head-site, personalized multimodal sensing of engagement for adaptive educational interfaces.