Rotation method improves tracking nonlinear data patterns in streams
Rotation-Based Subspace Tracking for Robust Kernel PCA on Streaming Data
Machine Learning
Summary
Data that changes over time or arrives bit by bit is hard to summarize quickly, especially when its patterns are complex and nonlinear. Traditional methods like PCA struggle with streaming and nonlinear data, and can be thrown off by unusual data points called outliers. The authors introduce a new way to update the data summary by rotating the estimated data space toward new information rather than just nudging it gradually. Their experiments show this rotation approach improves tracking accuracy and handles outliers better in streaming data.
What this means in practice
- •For data engineers: Improve real-time feature extraction on streaming data with nonlinear patterns and outliers by using rotation-based subspace updates instead of gradient descent.
- •For financial analysts: Enhance detection of evolving market factors in streaming financial data despite outliers, by employing robust kernel PCA with rotation updates.
Authors
Kris Lokere, John Fossaceca
Abstract
Machine learning models process large amounts of data, and Principal Component Analysis (PCA) is a widely used technique to reduce the dimensionality of the data and extract useful features. In practice, datasets often change over time (data drift) and/or arrive one sample at a time (streaming data), making it infeasible to process the entire dataset at once in batch mode. Real-world data also often contains nonlinear patterns, which traditional PCA cannot extract. Kernel PCA addresses this by implicitly mapping samples into a Reproducing Kernel Hilbert Space (RKHS). Raw data also often contains outliers, which can have an outsized effect on the estimated subspace unless the algorithm is made robust. However, existing online robust kernel PCA algorithms are designed to converge to a subspace that is assumed to be fixed, and gradient-descent-based updates lose their effectiveness at tracking further changes once this initial alignment is achieved. This paper introduces a rotation-based update mechanism, which updates the subspace estimate by rotating it toward each new incoming feature vector in Reproducing Kernel Hilbert Space, rather than relying on gradient descent alone. We present two complementary rotation strategies, and show that the extent of rotation can be moderated by a robust influence function to mitigate the effect of outliers. Through experiments on synthetic streaming data with a known ground-truth subspace, we show that per-sample rotations converge faster than gradient descent alone, demonstrating an effective mechanism for dynamically tracking a nonlinear subspace in streaming data.