Enhanced joint sparsity via iterative support detection

Joint sparsity has attracted considerable attention in recent years in many fields including sparse signal recovery in compressed sensing (CS), statistics, and machine learning. Traditional convex models suffer from the suboptimal performance though enjoying tractable computation. In this paper, we propose a new non-convex joint sparsity model, and develop a corresponding multi-stage adaptive convex relaxation algorithm. This method extends the idea of iterative support detection (ISD) from the single vector estimation to the multi-vector estimation by considering the joint sparsity prior. We provide some preliminary theoretical analysis including convergence analysis and a sufficient recovery condition. Numerical experiments from both compressive sensing and feature learning show the better performance of the proposed method in comparison with several state-of-the-art alternatives. Moreover, we demonstrate that the extension of ISD from the single vector to multi-vector estimation is not trivial. In particular, while ISD does not work well for reconstructing the signal channel sparse Bernoulli signal, it does achieve significantly improved performance when recovering the multi-channel sparse Bernoulli signal thanks to its ability of natural incorporation of the joint sparsity structure.

[1]  Jieping Ye,et al.  Multi-Task Feature Learning Via Efficient l2, 1-Norm Minimization , 2009, UAI.

[2]  M. Lai,et al.  The null space property for sparse recovery from multiple measurement vectors , 2011 .

[3]  Diego H. Milone,et al.  Wavelet shrinkage using adaptive structured sparsity constraints , 2015, Signal Process..

[4]  Hongtao Lu,et al.  A Fast Algorithm for Recovery of Jointly Sparse Vectors based on the Alternating Direction Methods , 2011, AISTATS.

[5]  Richard G. Baraniuk,et al.  Joint Sparsity Models for Distributed Compressed Sensing , 2005 .

[6]  Yuejie Chi,et al.  Joint sparsity recovery for spectral compressed sensing , 2013, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Helmut Bölcskei,et al.  Joint sparsity with different measurement matrices , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[8]  Søren Holdt Jensen,et al.  Joint sparsity and frequency estimation for spectral compressive sensing , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Taner Ince,et al.  Nonconvex compressed sensing with partially known signal support , 2013, Signal Process..

[10]  Yaru Fan,et al.  Cartoon-texture image decomposition via non-convex low-rank texture regularization , 2017, J. Frankl. Inst..

[11]  Chengcai Leng,et al.  Lp Regularization for Bioluminescence Tomography Based on the Split Bregman Method , 2016, Molecular Imaging and Biology.

[12]  Fernando José Von Zuben,et al.  Multi-task Sparse Structure Learning , 2014, CIKM.

[13]  Shuiwang Ji,et al.  SLEP: Sparse Learning with Efficient Projections , 2011 .

[14]  Jong Chul Ye,et al.  Compressive MUSIC with optimized partial support for joint sparse recovery , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[15]  Ting-Zhu Huang,et al.  Compressive Sensing via Nonlocal Smoothed Rank Function , 2016, PloS one.

[16]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[17]  Janghoon Yang,et al.  Utilization of Partial Common Information in Distributed Compressive Sensing , 2012, 2012 IEEE 75th Vehicular Technology Conference (VTC Spring).

[18]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[19]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[20]  Eric P. Xing,et al.  Heterogeneous multitask learning with joint sparsity constraints , 2009, NIPS.

[21]  Pramod K. Varshney,et al.  OMP Based Joint Sparsity Pattern Recovery Under Communication Constraints , 2013, IEEE Transactions on Signal Processing.

[22]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[23]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[24]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Tong Zhang,et al.  Analysis of Multi-stage Convex Relaxation for Sparse Regularization , 2010, J. Mach. Learn. Res..

[26]  Wotao Yin,et al.  Edge Guided Reconstruction for Compressive Imaging , 2012, SIAM J. Imaging Sci..

[27]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[28]  Lihua Xie,et al.  Exact Joint Sparse Frequency Recovery via Optimization Methods , 2014, 1405.6585.

[29]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]  Rabab Kreidieh Ward,et al.  Calibrationless Parallel Magnetic Resonance Imaging: A Joint Sparsity Model , 2013, Sensors.

[31]  Yoram Bresler,et al.  Subspace Methods for Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[32]  Samarendra Dandapat,et al.  Block sparsity-based joint compressed sensing recovery of multi-channel ECG signals , 2017, Healthcare technology letters.

[33]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[34]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Wei Liu,et al.  Discriminative Multi-instance Multitask Learning for 3D Action Recognition , 2017, IEEE Transactions on Multimedia.

[36]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[37]  R. Tibshirani The Lasso Problem and Uniqueness , 2012, 1206.0313.

[38]  Richard Bamler,et al.  Exploiting Joint Sparsity for Pansharpening: The J-SparseFI Algorithm , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Chuancai Liu,et al.  A one-dimensional slope detection approach , 2013, SpringerPlus.

[40]  Biao Hou,et al.  Joint Sparse Recovery With Semisupervised MUSIC , 2017, IEEE Signal Processing Letters.

[41]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[42]  Rung Ching Chen,et al.  An artificial bee colony algorithm for data collection path planning in sparse wireless sensor networks , 2013, International Journal of Machine Learning and Cybernetics.

[43]  Yang Fang,et al.  Visual music score detection with unsupervised feature learning method based on K-means , 2015 .

[44]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[45]  Mark W. Schmidt,et al.  GROUP SPARSITY VIA LINEAR-TIME PROJECTION , 2008 .

[46]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[47]  Zhu Han,et al.  Collaborative Spectrum Sensing from Sparse Observations in Cognitive Radio Networks , 2010, IEEE Journal on Selected Areas in Communications.

[48]  Urbashi Mitra,et al.  2008 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , 2008 .

[49]  Jong Chul Ye,et al.  Improving Noise Robustness in Subspace-Based Joint Sparse Recovery , 2011, IEEE Transactions on Signal Processing.

[50]  Wotao Yin,et al.  Sparse Signal Reconstruction via Iterative Support Detection , 2009, SIAM J. Imaging Sci..

[51]  Kaushik Mahata,et al.  A Robust Algorithm for Joint-Sparse Recovery , 2009, IEEE Signal Processing Letters.

[52]  Yilun Wang,et al.  Iterative Support Detection-Based Split Bregman Method for Wavelet Frame-Based Image Inpainting , 2014, IEEE Transactions on Image Processing.

[53]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[54]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[55]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[56]  Ya-Feng Liu,et al.  Lp-norm Regularization Algorithms for Optimization Over Permutation Matrices , 2016, SIAM J. Optim..

[57]  Anne Gelb,et al.  Composite SAR imaging using sequential joint sparsity , 2017, J. Comput. Phys..

[58]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[59]  Pierre Vandergheynst,et al.  Average Case Analysis of Multichannel Thresholding , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[60]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[61]  Jinchi Lv,et al.  High dimensional thresholded regression and shrinkage effect , 2014, 1605.03306.

[62]  Fady Alajaji,et al.  Compressed Sensing with Non-Gaussian Noise and Partial Support Information , 2015, IEEE Signal Processing Letters.

[63]  Jieping Ye,et al.  Multi-stage multi-task feature learning , 2012, J. Mach. Learn. Res..

[64]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[65]  Katya Scheinberg,et al.  Noname manuscript No. (will be inserted by the editor) Efficient Block-coordinate Descent Algorithms for the Group Lasso , 2022 .

[66]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  G. Obozinski Joint covariate selection for grouped classification , 2007 .