Low rank measures identified from few one dimensional projections
The Silhouette Operator: Identifiability of Low-Rank Measures from One-Dimensional Projections
Information TheoryMachine Learning
Summary
Recovering complex high-dimensional data from simpler, low-dimensional measurements is a big challenge. The authors study special types of structured data called low-rank signed measures, which can be broken down into simpler one-dimensional pieces. They show that by using just a certain number of carefully chosen one-dimensional projections, these complex measures can be fully identified. They also introduce a practical way to estimate such measures from real data that competes well with other popular methods.
What this means in practice
- •For statistical data analysts: Estimate complex joint distributions efficiently from lower dimensional projections for moderate data sizes and dimensions.
- •For signal processing engineers: Identify structured signals expressed as low-rank measures using minimal one-dimensional linear measurement data.
Authors
Robert A. Vandermeulen
Abstract
Structured recovery phenomena, such as restricted isometry properties in compressed sensing, have shown that high-dimensional objects can often be reconstructed from remarkably low-dimensional linear measurements. This work develops an analogous recovery framework for low-rank signed measures on $\mathbb{R}^2$, defined here as measures that can be expressed as finite sums of product measures with one-dimensional factors. The framework is based on linear operators, termed "silhouette operators," that map a measure to a fixed finite collection of one-dimensional linear pushforwards. The main results show that a suitably chosen collection of $2k$ projected marginals suffices to identify every compactly supported rank-$\le k$ signed measure, that this number is optimal, and that the projection directions cannot be chosen arbitrarily. The framework is also extended to higher-dimensional sums of product measures by establishing sufficient conditions under which collections of pairwise marginals identify the full model. Building on this framework, a computationally efficient estimator, termed "silhouette mixture estimation" (SME), is introduced for constructing a low-rank empirical measure from data by matching its one-dimensional projected marginals to the corresponding empirical marginals in Wasserstein distance. When combined with one-dimensional density estimators, SME yields an efficient nonparametric density estimator that performs strongly relative to a range of parametric, nonparametric, and deep-learning baselines in settings of moderate dimension and sample size.