Cross-subject emotion tracking from EEG and fNIRS signals improves accuracy
Modeling Shared and Individual Structure for Cross-Subject Continuous Affect Regression from EEG-fNIRS
Artificial Intelligence
Summary
Measuring emotions from brain signals usually works best when models see data from the same person they test on. This paper looks at a harder problem: predicting emotions for new people with no labeled data, just unlabeled brain signals while they watch the same videos as others. The authors separate the emotion signals into parts shared across people and individual differences, adjusting the shared part using a brain activity marker. Their method predicts emotion values better than existing approaches for new people without prior labels.
What this means in practice
- •For affective computing developers: Create emotion recognition systems that estimate continuous emotional states for new users without personalized labeled data, using shared and individual brain signal patterns.
- •For mental health monitoring teams: Develop tools that track emotional changes across individuals noninvasively with EEG and fNIRS data, improving generalization to unseen patients.
Authors
Xuan Wang, Bing Wang, Shuai Chang, Hao Yuan, Xinbo Qi, Xinyue Zhang
Abstract
Continuous, second-by-second valence-arousal estimation from physiological signals is typically studied in a subject-dependent setting, where the model sees labeled data from the same person it is later evaluated on. We study the harder zero-shot cross-subject variant on a synchronized EEG-fNIRS dataset: predict raw-scale ([1, 255]) valence and arousal trajectories for subjects whose labels the model never observes, given only their unlabeled EEG/fNIRS recordings while watching the same video stimuli as a disjoint set of training subjects. We decompose the affect trajectory into a structure shared across subjects who watch the same stimuli and an individual structure estimated for each test subject from a label-free EEG marker (alpha-band cross-channel synchrony), which rescales the shared trajectory around the scale midpoint. We validate the per-subject calibration mechanism on four independent axes: leave-one-subject-out correlation between the marker and each subject's true optimal gain, a functional-form comparison against non-linear alternatives, a repeated leave-4-out component ablation isolating each part of the pipeline's contribution, and a ceiling analysis bounding the remaining headroom for per-subject scaling. On held-out subjects, the model reaches an overall MAE of 25.96 / 22.80 across two evaluation batches (valence 21.94 / 19.6, arousal 29.98 / 26.0), well below EEGNet and ASAC-Net baselines reported for the same subject-independent split (raw scale score 60.6 and 55.0 respectively). We further report a systematic negative-result search across model architectures, feature representations, and prediction targets that found no signal able to improve on the single alpha-synchrony marker.