Summary
Intracranial electroencephalography (iEEG) records brain activity with electrodes inside the head, which helps in decoding what the brain is doing. But decoding iEEG signals across different hospitals and tasks is hard to compare because datasets and methods vary a lot. The authors created iMINDBench, a shared test with many tasks and consistent rules to fairly evaluate brain decoding models across different locations. They found popular models usually do better than simple baselines within their settings, but strong traditional methods still compete well between places. Adding more data from different people or hospitals only slightly boosts results, showing more work is needed to improve general brain signal decoding.
What this means in practice
- •For neural engineering teams: Compare and improve brain signal decoding methods reliably across multiple clinical datasets with a standardized test suite for iEEG.
- •For brain-computer interface developers: Evaluate how well decoding models trained on one hospital’s brain data generalize to other hospitals using a multi-institution benchmark.
Authors
Geeling Chau, Saba Hashemi, Yonghyeon Gwon, Eshani Patel, Jan DeWitt, Christopher Wang, Andrii Zahorodnii, Sabera J Talukder, Danny Dongyeop Han, Chun Kee Chung, Maryam M Shanechi, Yisong Yue
Abstract
Intracranial electroencephalography (iEEG) is widely used to record electrical activity directly from electrodes inside the human brain, making it an attractive modality for neural decoding. However, progress in iEEG decoding, especially toward general-purpose foundation models, remains difficult to measure reliably: datasets are task- or institution-specific, limiting evidence of generalization across tasks and recording environments, and preprocessing choices can strongly influence performance, making model improvements difficult to distinguish from preprocessing gains. Thus, we introduce iMINDBench, an iEEG Multi-Institution Neural Decoding Benchmark that evaluates models on a shared suite of fifteen decoding tasks across three naturalistic movie-watching datasets. The benchmark additionally defines standardized preprocessing tracks and fixed evaluation splits to support consistent model comparisons. Using iMINDBench, we find that the evaluated pretrained systems generally outperform baselines within their respective preprocessing tracks, while strong spectral baselines remain competitive across institutional datasets. In our scaling study, adding up to 25 times more supervised data from other subjects or institutions yields only small or task-dependent gains over within-session training. Together, these findings highlight the need for iEEG models that improve on strong preprocessing baselines and make more effective use of data across subjects and institutions. Project website: https://imindbench.github.io/