Self-concentration and value mixing cause attention issues in language models
It's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention
Computation and Language
Summary
Large language models sometimes focus too much on the first word in a sentence, creating problems called attention sinks and massive activations. This study shows that these issues are not caused by a technique called RoPE, but instead happen because the model’s attention focuses on itself due to the way it processes information over time. The authors found that limited mixing of values in attention layers also plays a part. These discoveries help us understand how language models work internally and could improve how they are made smaller and faster in the future.
Large Language ModelsAttention mechanismAttention sinkMassive activationsCausal maskValue mixingRoPEQuantizationTransformerInternal dynamics
Authors
Raito Kiya, Satoki Ohashi, Kosuke Sato, Go Kamoda, Ryosuke Takahashi, Yuji Yamamoto, Daiki Shiono, Keisuke Sakaguchi, Goro Kobayashi
Abstract
Large Language Models (LLMs) often exhibit "Attention Sink" (AS) and the accompanying "Massive Activations" (MAs) at the initial position of a sequence. These phenomena frequently co-occur, and MAs can pose challenges for low-bit quantization. In this study, we analyze the factors underlying AS and MAs that emerge at the initial position regardless of the token occupying it. Our experiments suggest that self-concentration of attention, resulting from the causal mask, and the subsequent Value-non-mixing in attention outputs contribute to AS and MAs. These findings provide new empirical evidence on the internal dynamics of LLMs, offering insights that may inform future quantization strategies and advance our understanding of the internal mechanisms of attention layers.