Transformer attention gating shapes how representations shrink or grow

Anisotropic Representation Dynamics in Transformers

Machine Learning

Summary

Transformers process information through layers where tiny differences in data representations can become smaller or larger. The authors studied how parts of the attention mechanism called query gating and key gating can change whether these differences shrink or grow as they move through a layer. They developed a theory that explains when and how attention patterns switch these behaviors using mathematical criteria. This helps understand how Transformers keep or change information internally and how sensitive they are to small changes.

What this means in practice

  • For transformer model developers: Identify design strategies to control when representation differences expand or contract in attention layers, improving model stability and interpretability.
  • For natural language processing engineers: Optimize attention gating configurations to enhance robustness against input perturbations and improve local information flow in transformer-based models.

Tested on simulated data.

Authors

Nguyen Do, My T. Thai

Abstract

Small differences between internal representations can shrink or grow as they pass through a Transformer layer, affecting how information is preserved and how sensitive the model is to perturbations. Yet it remains unclear when changes in attention weights can turn a representation difference that would otherwise shrink into one that grows. We develop a \textbf{Mixture-Gated Theory of Anisotropic Representation Dynamics} that separates self-attention into value transport, query gating, and key gating. Our analysis reveals how the geometry of attended representations constrains gating and how interactions between these mechanisms can reverse contraction induced by value transport. We formalize this phenomenon as gating-induced regime switching, where a representation difference contracts under fixed attention routing but expands under full attention. We derive a necessary and sufficient spectral criterion that certifies the existence or absence of switching and identifies switching directions without exhaustive search. Beyond existence, our theory characterizes maximum relative switching strength, certifies the severity of contraction that gating can overcome, and determines the largest dimension of a subspace in which every nonzero direction switches. Experiments across four Transformer architectures validate these predictions through analytical and finite-forward comparisons, revealing distinct and joint query--key gating mechanisms and nontrivial switching subspaces. Our results provide a mechanistic and spectral understanding of how attention routing can alter local representation dynamics.