Multimodal model improves brain signal analysis without retraining
A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis
Machine LearningArtificial Intelligence
Summary
Brain signals are important for studying the brain and diagnosing diseases, but existing models usually need to be retrained for each task or don’t understand the signals deeply enough. The authors created METIS, a large model trained on diverse brain signal data that can analyze many tasks without extra training. It performs better than previous general models across many tests and works well even with little new data. This opens up better ways to study and use brain signals.
What this means in practice
- •For clinical diagnostic teams: Use METIS to analyze brain signals accurately for multiple diagnostic tasks without retraining for each new analysis.
- •For neurotechnology developers: Integrate METIS for efficient interpretation of brain signal data across devices and applications, improving data efficiency and generalization.
Authors
Mingzhi Chen, Yiyu Gui, Guibo Luo, Yuchao Yang
Abstract
Brain signal analysis is essential for both neuroscience research and clinical diagnostics, yet current approaches face critical limitations. End-to-end models require task-specific retraining and exhibit limited generalization, while pre-trained models lack semantic depth and still depend on extensive fine-tuning. Meanwhile, general-purpose multimodal foundation models, though powerful in other domains, struggle to interpret brain signals due to representational misalignment and lack of domain knowledge. This study introduces a multimodal foundation model for zero-shot and multi-task brain signal analysis (METIS) through a unified language-signal alignment framework. METIS is pretrained on the largest and most diverse brain-signal corpus to date, comprising over 70,000 h of recordings from more than 11,000 subjects across 20 datasets. In a comprehensive zero-shot evaluation across 12 datasets, METIS outperformed the leading generalist model by over 20.9% in average accuracy. Remarkably, without any fine-tuning, METIS's performance matches or exceeds that of supervised, task-specific models. Furthermore, METIS demonstrates exceptional data efficiency and strong generalization, achieving an average AUROC advantage of over 16.0% in few-shot settings and 15.9% in cross-dataset transfer. This work establishes a new paradigm for general-purpose brain signal analysis, paving the way for next-generation neurotechnology.