Framework reveals how brain signal AI models make decisions
EEG-Xplain: Decoding Neural Black-Boxes of EEG Foundation Models
Artificial Intelligence
Summary
Brain signal decoding using AI models often works well but is hard to understand, which makes doctors hesitant to trust them. The authors designed a system that explains these AI models by showing which brainwave sensors and time points matter, as well as why certain brainwave frequencies influence decisions. They also use large language models to turn these technical insights into easy-to-understand language. Their tests show the explanations match known brain activity patterns and uncover some reliance on irrelevant signals. This framework helps make brain AI tools more trustworthy and interpretable.
What this means in practice
- •For clinical neurotechnologists: Provide transparent explanations for EEG-based AI decisions to support clinical diagnosis and increase clinician trust.
- •For biomedical device engineers: Identify critical EEG features used by AI models to improve design and validation of neural decoding devices.
Authors
Hansong Ma, Junxiao Wang
Abstract
EEG foundation models such as BIOT, LaBraM, and EEGMamba have achieved remarkable performance in neural signal decoding, but their black-box nature limits clinical trust and neuroscientific validation. We propose a unified attribution framework for interpreting EEG foundation models across heterogeneous architectures. The framework integrates gradient-, perturbation-, and activation-based explanation methods to analyze model behavior in spatial, temporal, and frequency dimensions. Spatially, it identifies critical EEG channels and visualizes their distributions using topographic maps. Temporally, it highlights decision-relevant signal segments through attribution heatmaps. In the frequency domain, it quantifies the contributions of canonical EEG rhythms via spectral perturbation analysis. To assess explanation reliability, we introduce a population-level evaluation combining Area Over the Perturbation Curve (AOPC) and cross-method consistency analysis. The framework further leverages Large Language Models (LLMs) to transform structured attribution outputs into natural-language reports, bridging low-level neural representations and high-level semantic reasoning. Experiments on benchmark datasets, including Mumtaz2016 and TUAB, demonstrate that the generated explanations are consistent with established neurophysiological markers, validating meaningful neural representations while exposing potential dependencies on artifacts and spurious patterns. The proposed framework provides a standardized approach for evaluating the interpretability, reliability, and physiological plausibility of EEG foundation models.