Class disentanglement in neural networks is mostly pairwise and depth-graded
Certified Topological Interaction in Neural Representations: Class Disentanglement Is Mostly Pairwise
Machine Learning
Summary
Understanding how neural networks separate different classes of data is important for improving their performance. This paper measures class separation in neural networks using a topological method that looks at how data points from different classes overlap at various scales. The authors find that class separation mostly happens between pairs of classes rather than groups of three or more, and that most of this separation occurs early in training. They also show that certain training choices affect separation differently, and that their topological measure relates to test accuracy but is less predictive than simpler probes.
What this means in practice
- •For machine learning engineers: Measure class separation reliably during training to assess model confusion between class pairs for better debugging and tuning.
- •For computer vision teams: Optimize data augmentation strategies by understanding their influence on class disentanglement and reducing overlap between visual categories.
Authors
Sushovan Majhi
Abstract
Class disentanglement (the separation of a representation's class-conditional point clouds along depth and over training) is usually read off descriptive curves. We measure it as certified topological interaction between labeled point clouds, using the recently introduced Intersection Euler Characteristic Profile: the Euler characteristic of the overlap of the clouds' ball unions as a function of scale, computed by one Alpha-complex sweep with no boundary-matrix reduction. Every number carries a test: exact permutation tests in both directions, a guarded separation certificate, and a paired test for the comparative claims applications make. Across 111 trained networks and 52,650 certified measurements, disentanglement is depth-graded and concentrated in the first epochs, and interaction quotients rank class pairs by confusability (Spearman rho=0.83), on par with cheap separability statistics. In a 96-model factorial population, augmentation is the one training choice that separates classes relative to chance; weight decay compresses the overlap without separating, and depth and width do nothing. The structural finding is one only a k-fold statistic can pose: the joint entanglement of a class triple sits below that of its strongest pair in 97% of triple-layer cells and 99.5% of deep cells, far below a measured null floor, in vision encoders and frozen language models alike. This pairwise dominance is a regularity, not a law: expected from the nesting of overlaps but not forced by geometry, present at initialization and in raw pixels, and manufactured in the last stage alone when a network memorizes random labels. The unnormalized profile mass predicts test accuracy (R^2=0.94), the quotient does not, and neither beats a linear probe. One lesson is reported in full: the paired test must use a scale-free statistic, or it certifies feature-norm dynamics as disentanglement.