More Data Cannot Break a Symmetry: Identifiability by Design
Machine Learning
Summary
The authors explain that when trying to match representations of stimuli without labeled data, the underlying geometry limits what can be identified. They show that some existing tests for this limitation actually give misleading results because of symmetrical patterns in the data. Using color examples, the authors demonstrate that choosing asymmetric designs helps avoid these problems and improves matching accuracy. They also highlight that avoiding these degeneracies can be done quickly before collecting any data. This approach applies to other regularly spaced types of stimulus features beyond color.
Authors
Jing Xu, Christopher Kanan
Abstract
Unsupervised representational alignment recovers a stimulus-by-stimulus correspondence from geometry alone, but the automorphism group of the stimulus geometry bounds what any such alignment can identify, before data exist. The obvious diagnostic for this degeneracy, the cheapest non-identity relabelling, ranks two published designs in the wrong order, because dense sampling creates near-duplicates whose transposition is nearly free. We turn this known invariance (Demetci et al., 2024) into a design-time diagnostic and intervention. In colour, where candidate geometries have closed form, we show that the failure is structural: sixty-four times the restart budget leaves a symmetric design unmoved while an asymmetric set at the same N recovers every time. Discriminating representational models and recovering a correspondence are essentially uncorrelated objectives (r = -0.02 over 3,000 subsets). Choosing nine colours by this diagnostic alone, without consulting any learned representation, moves all 93 model representations away from the degenerate point and cuts catastrophic alignment failures from 75% to 2% with the models, the layers, N and the solver all held fixed. The same risk arises wherever a regular design meets its candidate geometry's isometry group, including evenly spaced orientations, tones, or motion directions, and the check costs one function call before data collection.