New mathematical bound improves understanding of non-negative matrix factorization
Singularities of Non-negative Matrix Factorization and their application to Bayesian inference
Machine Learning
Summary
Non-negative matrix factorization (NMF) is a way to break down big tables of numbers into simpler parts, but it can be tricky because it has special mathematical challenges. The authors found a better way to describe these challenges using something called the real log canonical threshold (RLCT), which helps predict how well Bayesian methods will perform with NMF. Their improved formula gives a tighter estimate of this threshold, especially when the true internal size of the data is three or more. This can guide how accurately we expect Bayesian NMF to work in practice.
What this means in practice
- •For machine learning engineers: Use the improved RLCT bounds to better estimate Bayesian model errors in non-negative matrix factorization applications.
- •For signal processing teams: Improve confidence in Bayesian approaches for decomposing signals represented by non-negative matrix factorization based on tightened mathematical error bounds.
A theory result. No direct application yet.
Authors
Naoki Hayashi, Yota Maeda, Yasushi Esaki
Abstract
Non-negative matrix factorization (NMF) is a singular statistical model whose Bayesian asymptotics are governed by the real log canonical threshold (RLCT). We study the local geometry of the factorization map and derive an upper bound for the RLCT of NMF. Let $H$ be the model inner dimension and $H_0$ the non-negative rank of the true $M\times N$ matrix. Assuming that the true matrix admits a strictly positive factorization of inner dimension $H_0$ in the interior of the parameter domain, we prove, for smooth positive priors, that $λ\leq \{(H-H_0)\min(M,N)+H_0(M+N-H_0)\}/2$. This bound strictly improves the previous bound when $H_0\geq3$. The proof uses a local analytic normal form that separates independent linear coordinates from a residual matrix product. When $H=H_0$ also equals the ordinary rank of the true matrix, we obtain the exact value $λ=H_0(M+N-H_0)/2$. Under the standard assumptions of singular learning theory, these results bound the leading coefficients of the expected Bayesian generalization error and the Bayesian free energy.