Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization
2026-07-27 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied a method called Symmetric Non-negative Matrix Factorization (SymNMF) used to find hidden group patterns in large data matrices. Because SymNMF usually needs a lot of memory and computation, they redesigned it to work efficiently on very large datasets using GPUs and multi-node systems. They tested different algorithms and found a few that still work well even for extremely large matrices (up to a million by a million). Their work also compared soft factorization methods to simpler clustering, showing that the soft factorization remains important when data patterns become less clear. Overall, the authors demonstrated how to scale SymNMF for big data while maintaining accuracy.
Symmetric Non-negative Matrix FactorizationGPU computingExtreme Value TheoryPearson correlationAdaGradRMSpropStochastic Variance Reduced GradientSpectral analysisSpherical K-meansMatrix factorization
Authors
Lavinia Ghita, Dhruv Desai, Jake Goldberg, Roman Yokunda Enzmann
Abstract
Symmetric non-negative matrix factorization (SymNMF) recovers latent group structure from a dependence matrix, but its dense, quadratic-memory objective has confined prior work to moderate sizes. We present a large-scale GPU study of seven algorithm families (over 30 configurations) on absolute Pearson correlation and tail pairwise dependence matrices from Extreme Value Theory, two proxies for empirical risk-factor estimation on large portfolios. A trace-identity reformulation eliminates all $n \times n$ intermediates, so a single GPU reaches $n \approx 10^5$ and multi-node distribution scales to $n = 10^6$ and beyond. Under a two-phase protocol, eleven methods converge at moderate scale; six remain efficient enough at $n = 10^5$ (five AdaGrad-family plus ADMM), and five AdaGrad-family methods still converge at $n = 10^6$: AdaGrad, RMSprop, and three we introduce (Piecewise AdaGrad, Row-Stochastic SVRG, Block-SVRG AdaptGrow). At $n = 10^6$ the fastest solver tracks the matrix spectrum: Block-SVRG AdaptGrow wins on the flat, ill-conditioned tail-dependence spectrum, where its lower per-iteration cost decides a long factorization, and full-batch AdaGrad wins on the dominant-low-rank correlation spectrum, where the run is short. We also benchmark spherical K-means as a hard-label baseline: cheaper when angular cluster structure is present, yet provably degenerate once the matrix collapses toward a single common factor, where the soft factorization remains necessary.