Learning as Observable Matrix Dynamics: Diffusive Relaxations versus Phase Transitions
2026-06-29 • Machine Learning
Machine Learning
AI summaryⓘ
The authors present Observable Matrix Dynamics (OMD), a tool to study how neural networks change their internal understanding of data during training by looking at fixed-size distance matrices between inputs. OMD applies ideas from random matrix theory to detect subtle changes in these relationships that usual training loss measures can miss. They analyze patterns in these matrices across several experiments, identifying different geometric behaviors in the network’s representations. This approach provides a detailed view of the network’s learning process beyond simply measuring dimensionality.
Observable Matrix DynamicsRandom Matrix TheoryDistance MatrixNeural Network TrainingSpectral AnalysisLatent SpaceBogomolny--Bohigas--Schmit TheoryMulti-Dimensional ScalingEigenvaluesRepresentation Geometry
Authors
Igor Halperin
Abstract
Observable Matrix Dynamics (OMD) is a diagnostic framework that probes the dynamics of high-dimensional internal representations of inputs by a neural network via a fixed-size $N \times N$ distance matrix $M(t)$ on a held set of $N$ inputs. OMD uses methods of random matrix theory and particle dynamics to explore spectral reorganisations that are missed by scalar loss functions, but are informative of the training process. We read $M(t)$ against a perturbative ambient-versus-latent decomposition extending the Bogomolny--Bohigas--Schmit (BBS) theory of random distance matrices, with per-snapshot diagnostics for the top-of-spectrum band structure and ambient noise, trajectory-level observables linking snapshots, and a 3D MDS embedding (bottom-three eigenvectors) rendering training as a moving particle cloud. Across seven experiments, diffusive regimes lack stable top-of-spectrum band structure, while sharp endogenous or externally driven reorganisations produce stable fingerprints: consistent with smooth or product latent geometries in BBS-adjacent cases, and with finite-cluster or Fourier-soliton structures otherwise. OMD thus reads the geometric regime of a representation rather than reporting a single intrinsic dimension.