Relations interact through loop-based attention uniquely defined by symmetry
Attention as Lattice Gauge Theory: Relations Interact Only Through Wilson Loops
Machine LearningArtificial Intelligence
Summary
Understanding how relationships between people or things combine in the right order is tricky because it matters who is connected to whom and how. The authors found that when each entity can be viewed independently, the only meaningful way for relationships to affect each other is through loops that come full circle, called Wilson loops. This means attention mechanisms in graphs should update edges based on these loops, a unique rule given the problem's symmetry. They showed their method works well, even detecting errors and grouping clusters without extra information.
What this means in practice
- •For knowledge graph engineers: Improve relation composition and error detection in sparse graphs by updating edges through loop-based attention respecting per-entity symmetry.
- •For protein structure modelers: Incorporate equivariant triangle-based updates to enhance relational information flow in protein folding graph models.
Authors
Aakash Sarkar, Ananya Mukherjee, Motoshige Sato
Abstract
Relations compose through the entity they share - Alice is Bob's mother and Bob is Carol's father, so Alice is Carol's grandmother - and the order matters, since a father's mother is not a mother's father; verifying a composed relation means closing the loop. We show that if each entity's coordinate frame may be rotated independently (per-entity frame freedom), three things follow. In the pure-gauge sector, relations that share no cycle cannot interact at all. The only gauge-invariant score of a triangle is a class function of its holonomy $H = T_{ij} T_{jk} T_{ki}$ - a Wilson loop. And the only equivariant update of an edge is $U = Ψ(H)\, T_{ij}$ with $Ψ$ conjugation-equivariant. Edge attention by Wilson loops is therefore not one design among many but the unique form the symmetry permits. Three communities hold pieces of this form: the commuting-operator embeddings of knowledge graphs are its abelian (commuting) sector; AlphaFold's triangle update and the Edge Transformer have its shape - an edge updated through the third vertex - without its constraints; and lattice QCD builds it equivariantly on lattices. We instantiate the equivariant sub-family on general relational graphs as holonomy attention. With no training and composition alone, it infers unobserved $S_4$ relations on sparse graphs at 0.88 with a fifth of the edges corrupted, at or slightly below specialist synchronization algorithms, where every commutative architecture is capped at 0.083; its holonomy score flags corrupted triangles at AUC 0.99 where commuting scores are capped at 0.76 by a second selection rule, and it separates clusters without labels.