Nest encodes multiscale data clusters inside one embedding space
NEST: Nested Embedding Representation for Multiscale Clustering
Machine Learning
Summary
Data from the real world often groups together in small and big clusters simultaneously, but most methods only find one type of group or make a clear hierarchy. The authors introduce NEST, a method that learns all these layers of groups inside one single embedding space. This embedding lets you see coarser or finer clusters simply by looking at different parts of the representation. Even without showing any hierarchy during training, NEST can recover multiscale structures in different types of data like images, text, and biology.
What this means in practice
- •For data engineers: Organize complex datasets exhibiting layers of groups into a single embedding for more flexible and efficient clustering.
- •For machine learning practitioners: Design embeddings that inherently encode multiple granularities of cluster structure to improve downstream analysis or visualization across fields.
Authors
Zhanqi Zhang, Mikio Aoi, Gal Mishne
Abstract
Real-world data often exhibits cluster structure at multiple resolutions, yet existing methods typically learn a single partition or construct an explicit hierarchy. Inspired by Matryoshka Representation Learning, we introduce NEST, an unsupervised framework that encodes coarse-to-fine structure within a single embedding. NEST jointly optimizes clustering and separation across nested prefix subspaces, enabling progressively finer distinctions as embedding dimensionality increases, with rank and within-cluster regularization that keep the embedding from collapsing. Empirically, distances in the learned embedding follow the hierarchy distance more closely than those of the input, although the hierarchy is never observed during training. On synthetic hierarchies, NEST recovers multiscale structure given only the number of clusters per level. Experiments on vision, text, and single-cell data further demonstrate multiscale organization across real-world representations. By making clustering granularity an intrinsic property of the representation, NEST provides a unified approach to multiscale structure discovery.