Neural models and brain areas bind phonemes to word positions differently
Binding Phoneme Identity to Position in Neural Models and the Human Brain
Computation and Language
Summary
Understanding how the brain and computers keep track of the order of sounds in words is tricky. The authors looked at how neural networks and human brains connect phonemes (basic sounds) to their positions in words. They found that not just the strength but also the direction of brain signals changes with position, forming a spiral pattern. This insight helps explain how different sequences like /gum/ and /mug/ are recognized as different words.
What this means in practice
- •For speech recognition developers: Improve speech recognition models by incorporating spiral encoding of phoneme identity and position inspired by brain and model findings.
- •For neurotechnology engineers: Design brain-computer interfaces that decode speech by exploiting geometric patterns in auditory cortex responses to phoneme sequences.
Authors
Nafiseh Momeni, Malo Renaudin, Itzhak Fried, Yair Lakretz
Abstract
A fundamental challenge in cognitive science and neural computation is the binding problem. Compositional generalization can be achieved by factorizing representations into roles, the structural slots of a representation (e.g., serial position or syntactic function), and fillers, the contents occupying them (e.g., a phoneme or a word), which can be recombined freely to represent novel combinations. How neural systems implement role--filler binding, however, remains unknown. One of its simplest cases is phoneme sequencing, where phoneme identity (filler) must be bound to position (role) so that /gum/ and /mug/ are represented differently. A proposal from NLP, the `onion hypothesis' (Csordas et al., 2024), holds that the direction of the neural population state encodes token identity, while its magnitude encodes position. It was formulated for random token sequences; we extend it to natural language and test it in neural models and the human brain. We first evaluated it in an encoder-decoder trained to repeat English words, using representational geometry analyses and causal interventions. Magnitude alone proves incomplete: position is encoded by magnitude together with a systematic rotation of the state direction, a scheme we formalize as the `spiral hypothesis'. We then analyzed intracranial recordings from the auditory cortex of 20 participants listening to short sentences. Across superior temporal gyrus electrodes, the change in population response between consecutive phonemes decreases in magnitude as position advances, and its direction is aligned within consonants and within vowels but near orthogonal across the two classes, supporting the magnitude coding shared by both accounts. These results provide a geometric account of how neural systems bind phoneme identity to position, and show how neural-model hypotheses can yield testable predictions about human neural computation.