Structured sparse learning improves modeling with joint priors and stable tuning
A knowledge-driven nonconvex square-root regularization framework for joint group sparsity and structured smoothness
Machine Learning
Summary
Modeling complex data often needs special assumptions about groups and smooth changes. Most methods use simple loss functions and tend to shrink big signals too much, needing careful tuning that depends on unknown noise. The authors propose a new method that avoids needing noise estimates and can handle both group sparsity and smoothness at once, using a nonconvex approach. They show mathematically it works well and develop an efficient algorithm to solve it, demonstrating better performance on various datasets.
What this means in practice
- •For data scientists: Improve predictive models by jointly enforcing group-level sparsity and smoothness without tuning for noise levels.
- •For signal processing engineers: Enhance signal recovery methods by applying stable, nonconvex regularization that respects group and smooth signal structures.
Authors
Wanling Xie, Hu Yang
Abstract
Structured sparse learning is a leading paradigm for high-dimensional data modeling, as it enables rigorous integration of domain-validated structural priors into the learning process. Among these priors, pre-defined grouped structure and piecewise smoothness are ubiquitous in real-world applications. However, most existing structured regularization methods are built on squared loss and convex penalties, rarely jointly model these two complementary structural priors, and suffer from two key limitations. Their optimal tuning parameters are sensitive to the unknown noise level $σ$ due to the squared loss. Meanwhile, convex penalties induce inherent estimation bias and severe over-shrinkage of large magnitudes. To address these issues, we propose a knowledge-driven Nonconvex Structured Smooth Adjustment for Square-Root Regularization (NS$^3$R), which unifies the $σ$-free square-root loss with a joint nonconvex $\ell_{2,q}$ penalty ($0 < q < 1$) to simultaneously encode group sparsity and structured smoothness priors. Theoretically, we establish rigorous non-asymptotic error bounds for the NS$^3$R estimator under a mild $(2,q)$-group restricted compatibility condition. Computationally, we develop an efficient double-screened IRLS with thresholding (DS-IRLS-th) algorithm, which adopts a nested double-screening strategy to reduce computational cost and improve numerical stability, with provable global convergence for the resulting nonconvex optimization problem. Extensive experiments on synthetic and real-world data demonstrate that NS$^3$R consistently outperforms state-of-the-art methods in finite-sample settings.