Transformers reveal which attention heads carry redundant information
SVD-Attention: Efficiently Analyzing Attention Blocks in Transformers
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Transformer models use attention mechanisms where different 'heads' focus on various parts of input data. The authors developed SVD-Attention, a method that analyzes these heads to find which parts actually use all their capacity and which are redundant depending on the task. They found patterns of redundancy in certain model layers depending on the type of task, such as sentiment analysis or question answering. This insight helps identify parts of the model that could be simplified without significantly affecting performance. Their method works as an analysis tool and shows small improvements in efficiency when combined with pruning.
What this means in practice
- •For machine learning engineers: Identify less useful attention heads in transformer models to improve inference efficiency with minimal loss in accuracy.
- •For natural language processing teams: Optimize transformer architectures for specific tasks by focusing compute on layers and heads shown to be critical by SVD-Attention analysis.
Authors
Debopam Sanyal, Hongjie Chen, Riya Elizabeth Kuriyan, Arth Tak, Divya Kothandaraman, Akshay Mehra, Joshua Kimball, Alexey Tumanov
Abstract
Pretrained transformers spend a large share of their inference compute inside self-attention, yet the per-head query-key interaction is structurally low-rank by construction. We ask a diagnostic question: across layers and architectures, which heads actually use the full rank budget on real data, and which do not? We propose SVD-Attention, a post-hoc method that decomposes the per-head interaction matrix $M_h$ and scores every singular mode by both its spectral magnitude and its activation variance across tokens on a small calibration set. The score keeps the modes the head can amplify and that real data actually activates, and discards the rest; the resulting factorization is exact for the retained modes and needs no finetuning. Across seven pretrained models and five task types, the diagnostic finds a task-dependent pattern: redundancy sits in the early to middle layers for sentiment and image classification and for generation, and in the late layers for dense prediction and extractive question answering. It is also predictive: truncating the layers it flags costs at most $1.75$ F1, while truncating the same number of layers elsewhere in the same model costs up to $50$ F1. SVD-Attention is an analysis tool rather than a compression method: acting on it reduces inference FLOPs by $0.8$-$1.5\%$ on its own and by $8$-$11\%$ when composed with weight pruning, with $0$-$4\%$ change in task quality.