Block sparse spectral method improves data clustering and interpretation
Corrected Block Sparse Spectral Optimization: Exact Models, Finite Identification, and Local Dynamics
Artificial IntelligenceMachine Learning
Summary
The paper focuses on improving a mathematical tool that helps find important patterns in complex data by selecting groups of related features simultaneously. The authors clarify previous models and present exact versions that handle tricky cases like ties and zero values more reliably. They prove the method eventually identifies the key features exactly and behaves predictably nearby. Testing on artificial and real datasets shows their approach finds clearer, more meaningful clusters than traditional methods.
What this means in practice
- •For data scientists: Extract clearer, sparser components from complex high-dimensional datasets to improve clustering and interpretation.
- •For machine learning engineers: Enhance dimensionality reduction techniques in AI models by using improved block sparse spectral methods for better feature extraction.
Authors
Omid Shams Solari
Abstract
The block generalized-power method pairs threshold-based support selection with inexpensive Stiefel updates, but the standard block formulation conflates two nonequivalent objectives and leaves inactive components, threshold ties, and post-selection refitting ill-specified. We rebuild the model from the primal. Eliminating each loading column from the normalized loading-ball problem gives a sum of soft-thresholded Euclidean norms (A-SUM); the squared-threshold objective in common use arises from a quadratic lift (A-SQ). We prove both identities, exhibit a two-component instance whose maximizers disagree, and specify explicit conventions for zero-response columns, ties, rank deficiency, and mask-before-refit. For A-SQ we separate the historical full-polar iteration from a safeguarded Riemannian ascent and prove, for the latter, whole-sequence convergence via a sufficient-ascent lemma, a relative-error lemma, and the Kurdyka-Lojasiewicz property, which applies because the soft-threshold objective and the Stiefel constraint are semi-algebraic. A strict threshold margin then forces finite identification of the support and sign pattern. The post-identification map is ordinary power iteration on the active covariance for a rank-one hard threshold, and the Frechet derivative of a coupled polar map for component-specific supports; a radius-based safe-screening rule follows from the same local analysis. Planted-spike experiments match the predicted identification time and local rates. On modern-AI panels (GPT-2 residual stream, ViT CLS stream, Pitprops) the thresholded block-sparse components match dense PCA on reconstruction and improve on it in monosemanticity and class specialisation. On three native-label scRNA datasets (Zheng17 FACS-sorted PBMCs, 68k donor-A with FACS-correlated labels, Tabula Muris) A-SQ beats dense PCA on cluster-recovery AMI by +0.023 to +0.044 at matched sparsity.