Zero shot method improves eeg models with multimodal data
ZeroMAG: Zero-Shot Multimodal Adapter Generation for Plug-and-Play EEG Foundation Models
Machine Learning
Summary
EEG foundation models help understand brain signals but often ignore other body signals recorded alongside brain data. The authors present ZeroMAG, a method that improves these models by adding information from other body signals without needing extra training labels or adjustments. ZeroMAG works by generating adapters that enhance the model's understanding using unlabeled data. Testing shows it significantly improves accuracy compared to using brain signals alone and approaches the performance of fully supervised methods.
What this means in practice
- •For medical device developers: Enhance EEG devices by integrating extra body signals to improve diagnosis and monitoring without requiring new labeled data.$Commercial implications: Enables more accurate neuro-monitoring products that incorporate multiple physiological signals without extra training costs.
- •For wearable technology engineers: Implement zero-shot multimodal adapters to add new sensor data to existing EEG models, improving personalization on new users.
Authors
Yubo Wang, Jingying Ma, Xinliang Zhou, Yangxuan Zhou, Jiquan Wang, Sha Zhao, Yiyuan Yang, Yi Ding, Ziyu Jia, Chenyu Liu, Cuntai Guan
Abstract
EEG foundation models (EFMs) capture reusable knowledge from large-scale EEG data, while many EEG recordings also include companion physiological signals that provide complementary information beyond the EEG-only interface. The challenge is to preserve this pretrained knowledge while extending the EFM to heterogeneous multimodal recordings through an adaptation inferred from unlabeled target data. We introduce ZeroMAG, a zero-shot multimodal adapter generation framework that extends a frozen EEG encoder and prediction head using unlabeled target recordings, without target labels or target-side optimization. The target datasets are held out from all model training and selection in the ZeroMAG pipeline. ZeroMAG organizes companion modalities around a configuration-invariant adapter, constructs a modality-subject-task condition from unlabeled recordings and task context, and generates adapter weights in a function-constrained latent space learned from source adapters. Across six held-out target datasets and three EFM backbones, ZeroMAG improves balanced accuracy by 7.22 percentage points over EEG-only inference and 4.89 points over direct weight regression, while coming within 0.50 points of supervised multimodal adaptation on average. Ablations further show that removing functional supervision from either representation learning or conditional generation degrades generated-adapter performance, confirming the contribution of both components.