Deep attention parameters linked to outputs under smoothing functions

The Identifiability and Observability of Deep Normalized Attention

Machine Learning

Summary

This paper studies which parts of deep single-head attention models can be uniquely determined by looking only at their inputs and outputs. The authors prove that under certain smooth transformations, like the common softmax normalization, these models' scoring and value combinations are mostly identifiable except for some sign ambiguities. They also explore cases where parameters become less observable due to score collapses, giving precise mathematical descriptions of these situations. The work includes calculations of how sensitive these parameters are when inputs nearly collapse, helping to understand when and why numerical difficulties occur.

What this means in practice

  • For machine learning engineers: Design attention layers with better parameter recovery by knowing when scores become unobservable or parameters degenerate, improving model interpretability and debugging.
  • For numerical analysts: Assess and mitigate numerical sensitivity in deep attention models during near-collapse scenarios using the authors’ Jacobian decay and contact order results.

A theory result. No direct application yet.

Authors

Pranav Venkata Konda

Abstract

We study which parameters of deep, unmasked, single-head attention are determined by its input--output function. For known positive nonconstant real-analytic normalizers, the function generically determines the effective scores and combined value map up to the signs induced by even normalizers. This proves the real-analytic case of a conjecture of Henry--Marchetti--Kohn, including softmax. We then classify exceptional fibers under explicit normalizer conditions, identifying when collapse makes later scores unobservable, and establish sharp Taylor orders for local identification. Near simultaneous query/key collapse, we compute the complete native Jacobian decay spectrum on separating finite input banks. For common first nonconstant normalizer degree $k$, layer $i$ has contact order $2k3^{i-1}-1$, with exact multiplicities and kernel dimension. High-precision and automatic differentiation calculations illustrate the resulting loss of numerical sensitivity.
View PDFOpen arXiv