Nonreciprocal spherical attention improves compressed similarity searches

Nonreciprocal Spherical Attention and Retrieval: Sharp Coarsening Clocks and Certified Score Correction

Machine LearningInformation Retrieval

Summary

Finding similar items quickly in huge datasets is hard, especially when using compressed or simplified data. The authors develop a new math tool called nonreciprocal spherical attention to better handle how these searches work, especially when data points get grouped or compressed unevenly. They prove key properties about this method and introduce a correction technique that improves search accuracy. Tests on large image and visual feature databases show modest improvements in how well the system finds the closest matches.

What this means in practice

  • For machine learning engineers: Improve compressed nearest-neighbor search accuracy on large vector datasets with calibrated nonreciprocal attention corrections.
  • For search infrastructure teams: Refine semantic search systems using spherical normalization to reduce errors from indexing compression in billion-scale retrieval.

Authors

Faruk Alpay, Taylan Alpay

Abstract

We develop nonreciprocal spherical attention as a mathematical framework for coarsening and compressed nearest-neighbor retrieval. For two shrinking acute blocks whose centers have zero skew pairing and whose total mass ratio is subexponential in inverse temperature, we prove $\lim_{β\to\infty}β^{-1}\log T_β=1-\cosφ$, without restricting particle-count growth. An explicit three-body flow has two rigorously separated metastable exponents. For finite compression, mass centering yields a sharp $O(r^2)$ attention-output error bound at fixed temperature, with an explicit spherical-normalization correction. We formulate skew query-side score calibration as a strictly convex moment problem and derive a unique normal equation, residual-based optimization certificates, and bounded low-rank projections. A norm-aware margin objective calibrates native squared-distance rankings. Experiments use semantic retrieval data, one million visual descriptors with three codebook seeds, and a ten-million-vector FAISS IVF-PQ index with recomputed exact cosine ground truth. On SIFT1M, skew margin calibration raises native-L2 hit@10 from 0.8926 to 0.8948; symmetric calibration reaches 0.8996. The ten-million-vector index reaches 0.8827 hit@10 at 0.1794 ms per query in a four-thread batch benchmark, while validation selects no skew update. Exact streamed attention over ten million keys evaluates compressed memory on 1,024 held-out queries. Independent matrix solves, automatic differentiation, finite differences, interval arithmetic, full-flow tests, and nine Rocq-checked algebra lemmas support reproducibility. Directionality alone does not establish retrieval superiority.