OneEEG improves multi-task EEG understanding by combining signal and visual inputs
OneEEG: A Unified Signal-Visual Multimodal Language Model with Task-Aware Mixture of Projectors for Multi-Task EEG Understanding
Machine Learning
Summary
Understanding brain signals recorded by EEG machines is hard because these signals are interpreted differently for each task. The authors created OneEEG, a system that learns from both the raw EEG numbers and images of the EEG signals together, to better understand multiple EEG tasks at once. They also designed a special method to connect these different types of EEG data to a large language model for flexible reasoning. Their tests show OneEEG works better than existing models and help explain why showing EEG as images helps interpretation.
What this means in practice
- •For brain-computer interface developers: Improve EEG signal interpretation across diverse tasks by integrating visual and numerical EEG data for more accurate brain-computer interfaces.
- •For clinical neurologists: Assist in analyzing EEG recordings from different clinical assessments with a unified model that handles multiple EEG interpretation tasks.
Authors
Jiacheng Zhang, Xueyi Zhang, Xin Lu, Siqi Cai, Haizhou Li
Abstract
Electroencephalography (EEG) is widely used in clinical assessment, brain-computer interfaces, and cognitive-state analysis, yet EEG modeling remains fragmented across task paradigms. Recent multimodal large language models (MLLMs) provide a promising interface for flexible multimodal reasoning, but their ability to understand EEG inputs remains limited. In this work, we propose OneEEG, a unified EEG-MLLM framework for multi-task EEG understanding. By training on our curated OneEEG-Instruct, a multi-task EEG instruction-tuning dataset, OneEEG jointly models numerical EEG representations and visual EEG representations, and introduces a task-aware Mixture of Projectors (MoP) to align heterogeneous EEG features across different task families with the LLM embedding space. We further establish OneEEG-Bench, a benchmark for evaluating current MLLMs and our method on multi-task EEG understanding. Experiments show that existing MLLMs have limited ability to interpret EEG inputs, while OneEEG achieves stronger performance by integrating numerical and visual EEG representations with task-aware EEG-to-LLM projection. Extensive ablations and analyses further reveal the effects of visual EEG renderings, MoP variants, router behavior, and feature dispersion across EEG tasks.