Language models use different words to send and receive information
Rethinking Contextualization by Reinterpreting Attention Head Channels
Machine LearningComputation and Language
Summary
Language models read sentences by passing information between words to understand context. The authors found that some words carry a lot of information and act as sources, while others carry less and mainly receive information. They show that this selective sharing happens in specific ways, and reinterpreted the model's attention system as a set of channels that control how information flows. This new view helps explain how language models focus on important words to build better meaning representations.
What this means in practice
- •For natural language processing engineers: Design language models that route information more precisely between words, improving model interpretability and possibly efficiency.
- •For machine learning platform developers: Create diagnostic tools that represent attention heads in a continuous feature space for better model analysis and tuning.
Authors
Hakaze Cho, Haolin Yang, Zhun Sun, Naoya Inoue, Benjamin Heinzerling, Kentaro Inui
Abstract
Contextualization, the core operation of language modeling, transmits information across words to build sentence-specific word representations. Prior works mainly study contextualization, focusing on individual words and attention heads as a growing discrete dictionary, lacking a global view of their general behavior. Therefore, we propose a general principle: Globally, we find and estimate that different words carry different amounts of information, and less-informative words tend to absorb more contextual information. Specifically, these low-information words do not absorb contextual words uniformly, and finer-grained selectivity enables more precise routing to promote information transmission between matched words. Moreover, to find what mechanism causes such processing, we reinterpret attention heads as channels gated by their singular vectors and find that: (1) these singular vectors point to the hidden states of more informative words, allowing such words to write their information to others more strongly to act as information sources, and vice versa; and (2) these singular vectors can be viewed equally as hidden state features, enabling automated interpretation of attention heads beyond prior heuristic head discovery, also embedding heads into a continuous space rather than treating them as discrete, independent dictionary entries.