Automating Learner Assessment: Benchmarking Machine Learning and Deep Learning Models for EEG-Based Familiarity Prediction
2026-08-17 • Human-Computer Interaction
Human-Computer InteractionMachine Learning
AI summaryⓘ
The authors studied how brain signals (EEG) can be used to tell if someone is familiar with certain knowledge, like faces or math equations. They tested 15 different computer models to predict familiarity based on brain wave patterns. Their results show that common testing methods can make the models look better than they really are, so they used a stricter test to get a more realistic performance measure. They found certain brain waves (Gamma and Beta) in specific areas are important for recognizing familiarity. This work provides a reliable baseline for using EEG to monitor learning in education.
Electroencephalography (EEG)FamiliarityMachine LearningDeep LearningPower Spectral DensityGamma WavesBeta WavesCross-ValidationSHAP AnalysisCognitive Monitoring
Authors
Isuru Nanayakkara, Thilina Halloluwa
Abstract
Objective assessment of learning remains a fundamental challenge in education. Electroencephalography (EEG) provides a direct, non-invasive window into the neural correlates of knowledge acquisition, including cognitive familiarity. This study benchmarks fifteen machine learning (ML) and deep learning (DL) models for EEG-based familiarity prediction across two cognitive domains: faces (factual knowledge) and mathematical equations (conceptual knowledge). Using continuous EEG data from 23 participants, we extract spectral features (Power Spectral Density) across six frequency bands. We show that while standard stratified cross-validation yields artificially high classification performance (up to 0.9853 F1-score using CNN) due to temporal leakage across neighboring epochs, a rigorous trial-independent validation (Group K-Fold) drops the peak performance to 0.6038 F1-score (using CNN), which is still statistically significant above the 25% chance level. This highlights the critical necessity of trial-independent evaluation to avoid overestimating model generalizability. Furthermore, feature importance and SHAP analysis reveal that temporal and frontal Gamma and Beta oscillations are the most critical biomarkers for familiarity. This work establishes a realistic benchmark for EEG-based cognitive monitoring in educational technologies.