AI summaryⓘ
The authors studied how to better decode spoken words from brain signals recorded by EEG, which is normally very noisy and changes across different recording sessions. Instead of averaging trials, they used a method that compares brain responses to the same speech recorded in different sessions to make the model learn consistent features. They also added a technique called variational regularization to make the model's internal representations less narrow and more general. Their tests on a Japanese EEG dataset showed this combined method lowered the error rate of reconstructed speech text without reducing the quality of the speech features. The authors also showed that their approach made the model's internal signals less dependent on specific recording sessions.
EEGspeech reconstructionsignal-to-noise ratiointer-session variabilitycontrastive learningvariational regularizationencodermel-spectrogramcharacter error rate
Authors
Tomoaki Mizuno, Toru Nakashika
Abstract
Reconstructing heard speech from non-invasive electroencephalography (EEG) is challenging due to a low signal-to-noise ratio (SNR) and inter-session variability. While trial averaging improves the SNR, it is difficult to apply to continuous speech. We instead use repeated EEG responses to the same stimulus across different sessions as positive pairs for contrastive learning, and introduce variational regularization that, combined with this contrastive objective, keeps the encoder representation space broad. Experiments on a Japanese EEG dataset show that combining the session-invariant strategy with variational regularization improves the character error rate (CER) while maintaining mel-spectrogram reconstruction fidelity. Session probing confirms that the encoder representations achieve session-invariance.