Neural networks closely approximate linear maps between data spaces
Do Neural Networks Learn Structure-Preserving Maps? A Case Study in Latent-to-Hilbert Embeddings
Machine Learning
Summary
This paper looks at whether neural networks can learn mappings that keep the underlying structure of data intact when transforming from one form to another. The researchers used images from MNIST and converted them into angles representing quantum states, then trained networks to recover these angles from a compressed representation. They found that the learned mappings were nearly linear and preserved key directions and inner products very well. Classical kernel methods sometimes worked even better, suggesting simpler approaches may suffice in some cases.
What this means in practice
- •For machine learning engineers: Use kernel ridge regression as an alternative to MLPs for learning near-linear structure-preserving mappings between latent and feature spaces.
- •For quantum computing developers: Enhance encoding schemes by understanding how neural networks approximate mappings from compressed classical data to quantum-inspired representations.
Tested on one dataset.
Authors
Muhammad Adnan Shahzad
Abstract
We ask whether a neural network can learn a structure-preserving map from a compressed latent space to a Hilbert-space representation. Using an 8-dimensional autoencoder bottleneck on MNIST and $n$-qubit product-state targets from PCA-based angle encoding, we report four findings. Although the target angles are generated by a nonlinear sigmoid transformation of the latent projections, the resulting mapping is well approximated by a linear function over the observed latent distribution: linear regression from $z$ to the true target angles achieves $R^{2} = 0.98$, while regression to the MLP's recovered angles achieves $R^{2} = 0.91$. The learned map's primary direction is strongly aligned with the target-induced direction, with cosine similarity $0.989$, while remaining nearly orthogonal to the input's principal direction, with cosine similarity $0.002$. The map is genuinely rank-4: removing any singular direction degrades inner-product preservation by $2.5$--$5.9\times$ despite a singular-value spectrum with two dominant and two small values. The learned subspace does not coincide with the PCA basis used to construct the target, and different random seeds recover the same primary direction but diverge in higher ranks. Finally, kernel ridge regression with an RBF kernel outperforms a tuned MLP (IP error $0.0144$ vs.\ $0.0197$), suggesting that for approximately linear structure-preserving mappings, classical kernel methods may be a simpler and more effective alternative.