Bayesian Compressive Sensing of Sparse Signals with Unknown Clustering Patterns

We consider the sparse recovery problem of signals with an unknown clustering pattern in the context of multiple measurement vectors (MMVs) using the compressive sensing (CS) technique. For many MMVs in practice, the solution matrix exhibits some sort of clustered sparsity pattern, or clumpy behavior, along each column, as well as joint sparsity across the columns. In this paper, we propose a new sparse Bayesian learning (SBL) method that incorporates a total variation-like prior as a measure of the overall clustering pattern in the solution. We further incorporate a parameter in this prior to account for the emphasis on the amount of clumpiness in the supports of the solution to improve the recovery performance of sparse signals with an unknown clustering pattern. This parameter does not exist in the other existing algorithms and is learned via our hierarchical SBL algorithm. While the proposed algorithm is constructed for the MMVs, it can also be applied to the single measurement vector (SMV) problems. Simulation results show the effectiveness of our algorithm compared to other algorithms for both SMV and MMVs.

[1]  Florent Krzakala,et al.  Approximate message passing with restricted Boltzmann machine priors , 2015, ArXiv.

[2]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.

[3]  Bhaskar D. Rao,et al.  On the benefits of the block-sparsity structure in sparse signal recovery , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[5]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[6]  Yonina C. Eldar,et al.  Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals , 2007, IEEE Transactions on Signal Processing.

[7]  Cong Ling,et al.  Convolutional Compressed Sensing Using Deterministic Sequences , 2012, IEEE Transactions on Signal Processing.

[8]  Todd K. Moon,et al.  Detection of Amorphously Shaped Objects Using Spatial Information Detection Enhancement (SIDE) , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[10]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[11]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[12]  Vishal Monga,et al.  Adaptive matching pursuit for sparse signal recovery , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Bhaskar D. Rao,et al.  Recovery of block sparse signals using the framework of block sparse Bayesian learning , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Trac D. Tran,et al.  Iterative Convex Refinement for Sparse Recovery , 2015, IEEE Signal Processing Letters.

[15]  Bhaskar D. Rao,et al.  Extension of SBL Algorithms for the Recovery of Block Sparse Signals With Intra-Block Correlation , 2012, IEEE Transactions on Signal Processing.

[16]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[17]  Lawrence Carin,et al.  Bayesian Robust Principal Component Analysis , 2011, IEEE Transactions on Image Processing.

[18]  Matthew J. Beal Variational algorithms for approximate Bayesian inference , 2003 .

[19]  Rabab Kreidieh Ward,et al.  Distributed Compressive Sensing: A Deep Learning Approach , 2015, IEEE Transactions on Signal Processing.

[20]  Bhaskar D. Rao,et al.  Sparse Signal Recovery With Temporally Correlated Source Vectors Using Sparse Bayesian Learning , 2011, IEEE Journal of Selected Topics in Signal Processing.

[21]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[22]  Todd K. Moon,et al.  On the block-sparse solution of single measurement vectors , 2015, 2015 49th Asilomar Conference on Signals, Systems and Computers.

[23]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[24]  Hong Sun,et al.  Bayesian compressive sensing for cluster structured sparse signals , 2012, Signal Process..

[25]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[26]  Jun Fang,et al.  Two-Dimensional Pattern-Coupled Sparse Bayesian Learning via Generalized Approximate Message Passing , 2015, IEEE Transactions on Image Processing.

[27]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[28]  Yonina C. Eldar,et al.  From Theory to Practice: Sub-Nyquist Sampling of Sparse Wideband Analog Signals , 2009, IEEE Journal of Selected Topics in Signal Processing.

[29]  Daniel Hernández-Lobato,et al.  Generalized spike-and-slab priors for Bayesian group feature selection using expectation propagation , 2013, J. Mach. Learn. Res..

[30]  Yonina C. Eldar,et al.  Efficient sampling of sparse wideband analog signals , 2008, 2008 IEEE 25th Convention of Electrical and Electronics Engineers in Israel.

[31]  Junzhou Huang,et al.  Learning with structured sparsity , 2009, ICML '09.

[32]  David B. Dunson,et al.  Multitask Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[33]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[34]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[35]  Jie Ding,et al.  Robustness of orthogonal matching pursuit for multiple measurement vectors in noisy scenario , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[37]  Ole Winther,et al.  Bayesian Inference for Structured Spike and Slab Priors , 2014, NIPS.

[38]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[39]  David B. Dunson,et al.  Multi-task compressive sensing with Dirichlet process priors , 2008, ICML '08.

[40]  Xiao-Ping Zhang,et al.  Direction-of-arrival estimation using sparse variable projection optimization , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[41]  Jun Fang,et al.  Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals , 2015, IEEE Trans. Signal Process..

[42]  Zhi Chen,et al.  Support knowledge-aided sparse Bayesian learning for compressed sensing , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Bhaskar D. Rao,et al.  Sparse Bayesian learning using approximate message passing , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[44]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[45]  Yonina C. Eldar,et al.  Spectrum Sharing Radar: Coexistence via Xampling , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[46]  Alexandros G. Dimakis,et al.  Compressed Sensing using Generative Models , 2017, ICML.

[47]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[48]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[49]  D. Donoho,et al.  Basis pursuit , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[50]  Todd K. Moon,et al.  AMP-B-SBL: An algorithm for clustered sparse signals using approximate message passing , 2016, 2016 IEEE 7th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[51]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[52]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[53]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[54]  Philip Schniter,et al.  Efficient High-Dimensional Inference in the Multiple Measurement Vector Problem , 2011, IEEE Transactions on Signal Processing.

[55]  Todd K. Moon,et al.  Hierarchical Bayesian approach for jointly-sparse solution of multiple-measurement vectors , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[56]  Yonina C. Eldar,et al.  Xampling: Signal Acquisition and Processing in Union of Subspaces , 2009, IEEE Transactions on Signal Processing.

[57]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo in Practice: A Roundtable Discussion , 1998 .

[58]  Hong Sun,et al.  Compressive sensing for cluster structured sparse signals: variational Bayes approach , 2016, IET Signal Process..

[59]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[60]  Bernard Ghanem,et al.  ISTA-Net: Iterative Shrinkage-Thresholding Algorithm Inspired Deep Network for Image Compressive Sensing , 2017, ArXiv.

[61]  Chun-Liang Li,et al.  One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[62]  Kaushik Mahata,et al.  A Robust Algorithm for Joint-Sparse Recovery , 2009, IEEE Signal Processing Letters.

[63]  Albert Wang,et al.  The In-Crowd Algorithm for Fast Basis Pursuit Denoising , 2011, IEEE Transactions on Signal Processing.