EEG model tuning improvements vary by dataset with added modules
Dataset-Dependent Effects of Cross-Depth Aggregation and Soft-Routed Experts in EEG Foundation Model Fine-Tuning
Machine Learning
Summary
Different brainwave decoding tasks rely on different patterns over time and between channels. The authors tested if adding special attention and expert modules to a brainwave analysis model helps improve accuracy when fine-tuning it. They found that these additions sometimes help but can also hurt performance depending on the dataset. The new modules require much more computing power and memory, so their benefits are not consistent across tasks.
What this means in practice
- •For eeg system developers: Improve EEG decoding models by selectively adding attention and expert modules depending on the dataset characteristics to balance performance and resource demands.
- •For wearable device engineers: Design brainwave monitoring devices with awareness that enhanced decoding modules may increase computational overhead and should be chosen based on target signals.
Authors
Mingyang Jiang, Yamin Li, Daniel Moyer, Fan Ma, Hua Xu, Catie Chang
Abstract
EEG decoding tasks can rely on different temporal dynamics and cross-channel relationships. We test whether specialized modules improve a fully fine-tuned EEG foundation model by augmenting CBraMod with cross-depth Attention Residuals (AttnRes) and two soft-routed expert banks. Across matched three-seed experiments on FACED, ISRUC, SEED-V, and PhysioNet-MI, the complete model changes mean balanced accuracy relative to full fine-tuning by -0.12, +1.27, +0.77, and -1.27 points, respectively. AttnRes alone improves mean balanced accuracy on three datasets, whereas adding experts on top of AttnRes helps only FACED and SEED-V. These gains come with substantial overhead: AttnRes requires 2.11 to 2.88x runtime and 1.78 to 2.67x memory, while the complete model requires 2.41 to 3.04x runtime and 1.86 to 2.85x memory. Overall, the added modules produce dataset-dependent, sometimes opposing effects rather than consistent gains over full fine-tuning.