Unsupervised Latent Space Alignment with Hyperspherical Geodesic Matching
Machine Learning
Summary
The authors study how different neural networks trained on the same data create similar hidden patterns, but these patterns aren’t directly compatible. They introduce a new method called HGA that aligns these hidden spaces by focusing on their geometric shape instead of relying on matching example pairs. This method can work with little or no direct supervision and performs well in tasks like connecting different models or linking multilingual word meanings. Essentially, the authors show it’s possible to match hidden representations based on their overall geometric structure alone.
neural networkslatent spacelatent geometrymodel alignmentunsupervised learningmodel stitchingword embeddingshyperspherical Gaussiangeometric transformationmultilingual embeddings
Authors
Cameron Ryan, Vivek Sivaraman Narayanaswamy, Kowshik Thopalli, Shusen Liu
Abstract
Independently trained neural networks tend to encode the same data with similar latent geometries. These latent geometries are not directly compatible, yet they can be nearly the same up to some class of transformations. While there exists many methods for alignment between different latent spaces, it is typically done using a set of shared sample correspondences, known as anchors. This leaves a fundamental question: are the geometric signatures of different latent spaces representing similar data sufficient to recover an alignment between them? To that end, we introduce HGA (Hyperspherical Gaussian Alignment), a method that directly optimizes a transformation between two latent spaces by maximizing a geometric measure of "fit" between them. Since it is driven by the geometry of the latent spaces rather than paired data, HGA can operate in both an unsupervised and weakly supervised regime. On tasks such as model stitching or multilingual word embedding correspondence recovery, HGA manages to match supervised results with minimal or no supervision.