Spectral Stability of Pseudoinverse-Based Extreme Learning Machine
2026-07-09 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied how a common method called Extreme Learning Machine (ELM) calculates its output weights using a mathematical tool called the pseudoinverse. They found that the smallest singular value of a key matrix controls how much small errors can get bigger in the results. By comparing different ways to compute the pseudoinverse, they showed that traditional SVD-based methods handle tricky cases better than newer iterative methods. Overall, the authors demonstrate that the stability of ELM largely depends on certain properties of the hidden layer matrix.
Extreme Learning MachineMoore-Penrose pseudoinverseSingular Value Decomposition (SVD)Singular valuesCondition numberHidden layer matrixNumerical stabilityHyperpower methods
Authors
Bich Van Nguyen, Ngoc Anh Khong
Abstract
Extreme Learning Machine (ELM) computes output weights analytically using the Moore-Penrose pseudoinverse. Although this leads to fast training, its numerical stability depends strongly on the conditioning of the hidden layer matrix. This paper studies pseudoinverse-based ELM from a spectral perspective. We show that the smallest singular value governs perturbation amplification in the output weights, while the condition number provides a quantitative measure of hidden-layer instability. We compare SVD-based pseudoinverse computation with iterative hyperpower methods and discuss width-dependent conditioning through a random feature interpretation. Experiments on synthetic matrices and ELM benchmarks show that SVD-based methods remain the most reliable under ill conditioning, while iterative methods are more sensitive to spectral properties. The results suggest that ELM stability is fundamentally governed by the singular value structure of the hidden layer matrix.