Papers for

brain computer interface teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

FMRI brain signals decode natural speech using multimodal time models

Decoding Naturalistic Speech from fMRI with Temporal Modeling and Multimodal Alignment

Abstract: Reconstructing what a person perceives from their brain activity is a stringent test of whether neural recordings carry meaningful information. Naturalistic speech offers a rich testbed for this challenge: it unfolds over multiple timescales, but fMRI's noisy, temporally smeared measurements make this structure hard to recover. We introduce NOEMA, a temporal convolutional decoder that aligns windows of consecutive fMRI measurements with multimodal stimulus embeddings. These embeddings combine complementary text, audio, and speech representations of the same stimulus segment, enabling its retrieval directly from brain activity. On two naturalistic speech datasets, Cephalonauts One and LeBel, NOEMA retrieves the correct segment among 2,000 candidates within the top 10 for more than 42% of held-out queries. This substantially outperforms baseline decoders, an advantage that persists when the candidate pool is extended with thousands of distractors absent from training. Controlled ablations establish the contributions of temporal modeling and complementary stimulus representations.

Thu 15 OctMachine LearningArtificial Intelligence
The gist
Figuring out what speech a person hears from brain scans is very difficult because brain signals change slowly and can be noisy. The authors developed NOEMA, a system that links brain activity patterns to speech segments by combining text, sounds, and speech features over time. Tested on two real speech datasets, NOEMA could pick the right sentence from thousands of choices more than 42% of the time, much better than previous methods. This shows brain scans contain detailed info about natural speech that can be decoded using clever time-based and multi-sense models.
Open → 2610.22049v1

Intracortical motor decoder adapts across sessions with fixed network weights

Association profile conditioning in a set-temporal transformer for cross-session intracortical motor decoding

Abstract: Intracortical motor decoders degrade across sessions because the set of recorded units changes and persisting units can alter how their firing relates to behavior. Most existing methods update network weights on each new session or rely on unlabeled activity, which does not directly reveal such changes. We present APST, an Association Profile-conditioned Set-Temporal transformer that adapts to new sessions with all network weights frozen. From a few labeled calibration trials, APST summarizes how each unit's firing relates to behavior in a four-dimensional association profile computed in closed form. The profiles condition a set-attention encoder that accepts any number and order of units, followed by a causal transformer for streaming decoding. On held-out DANDI688 sessions from two monkeys, APST reaches velocity $R^2$ of $0.78$ and $0.81$, versus $0.40$ and $0.58$ for a variant that uses neural activity alone, and matches or exceeds an RNN fine-tuned on the same trials. On FALCON private held-out evaluation, it attains $R^2$ of $0.65$, $0.42$, and $0.44$ on M1, M2, and H1.

Wed 30 SeptArtificial Intelligence
The gist
Intracortical devices that decode brain signals to control movements usually lose accuracy over time because the neurons they record from change or behave differently. The authors present a method called APST that adjusts to these changes without retraining the whole model, by analyzing how each neuron’s activity relates to movement during a few short calibration periods. This approach uses a transformer model that can handle varying numbers and orders of neurons and improves accuracy in predicting movement velocity across multiple sessions. APST outperforms methods that rely on neural data alone or require fine-tuning with the same calibration data.
Open → 2609.39080v1