Pullback geometry improves interpolation for multimodal data sets
Beyond Unimodal Bases: Pullback Geometry for Multimodal Data
Machine Learning
Summary
Many data methods assume data points lie on a single smooth shape, which works poorly for data with several distinct groups or patterns. The authors propose a new way to understand the geometry of data that fits multiple overlapping shapes using a mixture of Gaussian models. This method improves how data points are connected and interpolated along likely paths within these complex structures. They demonstrate better performance on synthetic, image, and handwritten digit data, showing clearer, more realistic transitions between data points. This advances tools for handling complicated, multimodal data beyond simple assumptions.
What this means in practice
- •For computer vision teams: Generate smoother and more realistic transitions between images from multiple viewpoints by better modeling complex data shapes.
- •For machine learning engineers: Improve interpolation and representation in multimodal data with adaptive mixture learning for enhanced data synthesis and reconstruction.
Authors
Honglei Brinkmann, Lucas Ng, Georgios Batzolis, Mark Girolami, Carola-Bibiane Schönlieb, Willem Diepeveen
Abstract
Data-driven Riemannian geometry provides nonlinear interpolation and geometric representations of high-dimensional data. For these operations to be statistically meaningful, paths between observations should preferentially traverse high-likelihood regions. Existing scalable pullback constructions typically use a unimodal Gaussian latent distribution, assuming that the data reside close to a single manifold. For multimodal data, mapping separated modes or local structures into one Gaussian region can require substantial transport deformation and compromise the resulting geometry. We introduce a pullback geometry for data supported on mixtures of manifolds. Using a latent Gaussian mixture, we define its Riemannian metric as the matrix square of the responsibility-weighted expected component precision. The metric is smooth and positive definite and recovers the existing Gaussian construction in the single-component limit. For structured overlapping mixtures, we establish conditions under which the log-density is concave along geodesics, providing a formal connection between the proposed geometry and paths through high-likelihood regions, and derive the corresponding local curvature relations. We instantiate this geometry in a normalizing flow with adaptive mixture learning, allowing the number of active components to emerge from the data and supporting component-wise reconstruction and local effective-dimension estimation. Experiments on synthetic geometric data, a controlled multi-view image setting with a known reference trajectory, and MNIST show reduced transport distortion, competitive path support, close reference-trajectory recovery, and improved interpolation realism. These results extend scalable pullback geometry beyond datasets that reside close to a single manifold while retaining tractable and interpretable local structure.