High dimensional pca improved by efficient subsampled subspace method
SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
Machine Learning
Summary
Finding patterns in complex, high-dimensional data can be hard and slow using traditional PCA. The authors showed that a combination of several directions in the data can reveal key signals before each individual pattern becomes clear. They developed SuperPCA, a faster and more accurate method that uses fewer measurements by focusing on important parts of the data. This method works especially well when the signals are spread out sparsely. SuperPCA improves accuracy up to ten times compared to regular PCA when finding multiple patterns.
What this means in practice
- •For signal processing engineers: Extract key signal directions from noisy high-dimensional sensor data efficiently using fewer data points.
- •For data acquisition teams: Reduce data collection costs by sampling only a small subset of high-dimensional coordinates while maintaining accurate signal recovery.
Authors
Irina-Beatrice Haas, Maike Meier, Yuji Nakatsukasa, Taejun Park
Abstract
Principal component analysis (PCA) is a fundamental tool to reduce the dimensionality of the data in many applications. PCA finds a few signal directions that contain most of the variability of the data by computing the eigenvectors of the sample covariance matrix. In this work, we focus on the spiked covariance model, in which the data vectors are defined by a few orthogonal signals plus an isotropic Gaussian noise, and our goal is to estimate one or more of the leading signals. Our main theoretical finding is that the subspace spanned by several leading eigenvectors of the sample covariance matrix contains significant information about the desired signals long before the individual eigenvectors converge to the population principal components. To prove this, we derive a posteriori bounds for the angle between the subspace spanned by the desired population signals and the subspace obtained from the sample using perturbation theory for singular vectors. This leads to a new algorithm, SuperPCA (SUbsPace subsamplER PCA), which capitalizes on an approximate eigenspace of the sample covariance matrix to find the leading signals far more efficiently and accurately than classical PCA in the high-dimensional, multi-signal setting. SuperPCA exploits only a small number of subsampled coordinates of the data, which can lead to tremendous savings in data acquisition cost, especially when the signals are approximately sparse. For the same number of measurements, SuperPCA can offer a factor $10$ improvement in accuracy compared to the classical PCA method.