Blind cluster structured sparse signal recovery: A nonconvex approach

We consider the problem of recovering a sparse signal when its nonzero coefficients tend to cluster into blocks, whose number, dimension and position are unknown. We refer to this problem as blind cluster structured sparse recovery. For its solution, differently from the existing methods that consider the problem in a statistical context, we propose a deterministic neighborhood based approach characterized by the use both of a nonconvex, nonseparable sparsity inducing function and of a penalized version of the iterative ?1 reweighted method. Despite the high nonconvexity of the approach, a suitable integration of these building elements led to the development of MB-NFCS (Model Based Nonlinear Filtering for Compressed Sensing), an iterative fast, self-adaptive, and efficient algorithm that, without requiring any information on the sparsity pattern, adjusts at each iteration the action of the sparsity inducing function in order to strongly encourage the emerging cluster structure. The effectiveness of the proposed approach is demonstrated by a large set of numerical experiments that show the superior performance of MB-NFCS to the state-of-the-art algorithms. HighlightsWe focus on recovering cluster structured sparse signals from few acquisitions without any information on the signal structure.We consider a compressed sensing approach with a nonconvex nonseparable neighborhood based sparsity inducing function.We solve the corresponding constrained nonconvex minimization problem by integrating an IRl1 scheme into the penalization approach.The convergence of the iterative algorithm to a local minimum of the original problem is guaranteed.The local nature of the approach allows the algorithm to learn the unknown signal structure during the reconstruction process.

[1]  Wenze Shao,et al.  Nonconvex Compressed Sampling of Natural Images and Applications to Compressed MR Imaging , 2012 .

[2]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[3]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[4]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[5]  N. Gould On the convegence of a sequential penalty function method for constrained minimization , 1989 .

[6]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[7]  Mark A. Davenport,et al.  Random Observations on Random Observations: Sparse Signal Acquisition and Processing , 2010 .

[8]  Ilker Bayram,et al.  Mixed norms with overlapping groups as signal priors , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Yue M. Lu,et al.  Sampling Signals from a Union of Subspaces , 2008, IEEE Signal Processing Magazine.

[10]  E.J. Candes Compressive Sampling , 2022 .

[11]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[12]  Damiana Lazzaro,et al.  An Iterative $L_{1}$-Based Image Restoration Algorithm With an Adaptive Parameter Estimation , 2012, IEEE Transactions on Image Processing.

[13]  C EldarYonina,et al.  Block-sparse signals , 2010 .

[14]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[15]  F. Bach,et al.  Optimization with Sparsity-Inducing Penalties (Foundations and Trends(R) in Machine Learning) , 2011 .

[16]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[17]  Ivan W. Selesnick,et al.  Translation-invariant shrinkage/thresholding of group sparse signals , 2013, Signal Process..

[18]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[19]  Bhaskar D. Rao,et al.  Extension of SBL Algorithms for the Recovery of Block Sparse Signals With Intra-Block Correlation , 2012, IEEE Transactions on Signal Processing.

[20]  Jieping Ye,et al.  Efficient Methods for Overlapping Group Lasso , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  James P. Reilly,et al.  Minimizing Nonconvex Functions for Sparse Vector Reconstruction , 2010, IEEE Transactions on Signal Processing.

[22]  Armando Manduca,et al.  Highly Undersampled Magnetic Resonance Image Reconstruction via Homotopic $\ell_{0}$ -Minimization , 2009, IEEE Transactions on Medical Imaging.

[23]  Damiana Lazzaro,et al.  A fast algorithm for nonconvex approaches to sparse recovery problems , 2013, Signal Process..

[24]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[25]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[26]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[27]  Lorenzo Rosasco,et al.  A Primal-Dual Algorithm for Group Sparse Regularization with Overlapping Groups , 2010, NIPS.

[28]  Charles A. Micchelli,et al.  A Family of Penalty Functions for Structured Sparsity , 2010, NIPS.

[29]  Jieping Ye,et al.  Efficient Sparse Group Feature Selection via Nonconvex Optimization , 2012, ICML.

[30]  Babak Hassibi,et al.  On the reconstruction of block-sparse signals with an optimal number of measurements , 2009, IEEE Trans. Signal Process..

[31]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[32]  Ivan W. Selesnick,et al.  Sparse Signal Estimation by Maximally Sparse Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[33]  Alfred O. Hero,et al.  Multidimensional Shrinkage-Thresholding Operator and Group LASSO Penalties , 2011, IEEE Signal Processing Letters.

[34]  P. Zhao,et al.  The composite absolute penalties family for grouped and hierarchical variable selection , 2009, 0909.0411.

[35]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[36]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[37]  Christian Jutten,et al.  Sparse Recovery using Smoothed ℓ0 (SL0): Convergence Analysis , 2010, ArXiv.

[38]  Ivan W. Selesnick,et al.  Group-Sparse Signal Denoising: Non-Convex Regularization, Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[39]  Hong Sun,et al.  Bayesian compressive sensing for cluster structured sparse signals , 2012, Signal Process..

[40]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[41]  Richard Baraniuk,et al.  Recovery of Clustered Sparse Signals from Compressive Measurements , 2009 .

[42]  Junzhou Huang,et al.  Learning with structured sparsity , 2009, ICML '09.

[43]  Rick Chartrand,et al.  Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[44]  Bruno Torrésani,et al.  Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients , 2009, Signal Image Video Process..

[45]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[46]  Brendt Wohlberg,et al.  A nonconvex ADMM algorithm for group sparsity with sparse groups , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[47]  Donald Goldfarb,et al.  2 A Variable-Splitting Augmented Lagrangian Framework , 2011 .

[48]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[49]  Mohamed-Jalal Fadili,et al.  Group sparsity with overlapping partition functions , 2011, 2011 19th European Signal Processing Conference.

[50]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[51]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[52]  Srikantan S. Nagarajan,et al.  Iterative Reweighted l1 and l2 Methods for Finding Sparse Solution , 2016 .

[53]  Bhaskar D. Rao,et al.  Latent Variable Bayesian Models for Promoting Sparsity , 2011, IEEE Transactions on Information Theory.