Forest Sparsity for Multi-Channel Compressive Sensing

In this paper, we investigate a new compressive sensing model for multi-channel sparse data where each channel can be represented as a hierarchical tree and different channels are highly correlated. Therefore, the full data could follow the forest structure and we call this property forest sparsity. It exploits both intra- and inter- channel correlations and enriches the family of existing model-based compressive sensing theories. The proposed theory indicates that only O(Tk+log(N/k)) measurements are required for multi-channel data with forest sparsity, where T is the number of channels, N and k are the length and sparsity number of each channel, respectively. This result is much better than O(Tk+Tlog(N/k)) of tree sparsity, O(Tk+klog(N/k)) of joint sparsity, and far better than O(Tk+Tklog(N/k)) of standard sparsity. In addition, we extend the forest sparsity theory to the multiple measurement vectors problem, where the measurement matrix is a block-diagonal matrix. The result shows that the required measurement bound can be the same as that for dense random measurement matrix, when the data shares equal energy in each channel. A new algorithm is developed and applied on four example applications to validate the benefit of the proposed model. Extensive experiments demonstrate the effectiveness and efficiency of the proposed theory and algorithm.

[1]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.

[2]  Amir Said,et al.  Wavelet compression of medical images with set partitioning in hierarchical trees , 1996, Medical Imaging.

[3]  Nick G. Kingsbury,et al.  Convex approaches to model wavelet sparsity patterns , 2011, 2011 18th IEEE International Conference on Image Processing.

[4]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[5]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[6]  Vivek K Goyal,et al.  Multi‐contrast reconstruction with Bayesian compressed sensing , 2011, Magnetic resonance in medicine.

[7]  Zhu Han,et al.  Collaborative Spectrum Sensing from Sparse Observations in Cognitive Radio Networks , 2010, IEEE Journal on Selected Areas in Communications.

[8]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[9]  Junzhou Huang,et al.  Fast multi-contrast MRI reconstruction. , 2014, Magnetic resonance imaging.

[10]  Philip Schniter,et al.  Compressive Imaging Using Approximate Message Passing and a Markov-Tree Prior , 2010, IEEE Transactions on Signal Processing.

[11]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[12]  Junzhou Huang,et al.  Learning with structured sparsity , 2009, ICML '09.

[13]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[14]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[15]  Xi Chen,et al.  Graph-Structured Multi-task Regression and an Efficient Optimization Method for General Fused Lasso , 2010, ArXiv.

[16]  Philip Schniter,et al.  Efficient High-Dimensional Inference in the Multiple Measurement Vector Problem , 2011, IEEE Transactions on Signal Processing.

[17]  Junzhou Huang,et al.  Exploiting both intra-quadtree and inter-spatial structures for multi-contrast MRI , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[18]  Xiaohui Chen,et al.  A Two-Graph Guided Multi-task Lasso Approach for eQTL Mapping , 2012, AISTATS.

[19]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[20]  G. Lorentz,et al.  Constructive approximation : advanced problems , 1996 .

[21]  Junzhou Huang,et al.  Efficient MR image reconstruction for compressed MR imaging , 2011, Medical Image Anal..

[22]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[23]  David B. Dunson,et al.  Multitask Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[24]  Junzhou Huang,et al.  The benefit of tree sparsity in accelerated MRI , 2014, Medical Image Anal..

[25]  Junzhou Huang,et al.  The Benefit of Group Sparsity , 2009 .

[26]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[27]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[28]  Dimitris N. Metaxas,et al.  Structured sparsity: theorems, algorithms and applications , 2011 .

[29]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[30]  Shiqian Ma,et al.  An efficient algorithm for compressed MR imaging using total variation and wavelets , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Jianwei Ma,et al.  Single-Pixel Remote Sensing , 2009, IEEE Geoscience and Remote Sensing Letters.

[32]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[33]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[34]  Junzhou Huang,et al.  Learning with dynamic group sparsity , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Junzhou Huang,et al.  Calibrationless Parallel MRI with Joint Total Variation Regularization , 2013, MICCAI.

[36]  Armando Manduca,et al.  Medical image compression with set partitioning in hierarchical trees , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[37]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[38]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[39]  Rabab Kreidieh Ward,et al.  Compressive color imaging with group-sparsity on analysis prior , 2010, 2010 IEEE International Conference on Image Processing.

[40]  A. Majumdar,et al.  Joint reconstruction of multiecho MR images using correlated sparsity. , 2011, Magnetic resonance imaging.

[41]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[42]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[43]  Junfeng Yang,et al.  A Fast Alternating Direction Method for TVL1-L2 Signal Reconstruction From Partial Fourier Data , 2010, IEEE Journal of Selected Topics in Signal Processing.

[44]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[45]  Kai Siedenburg,et al.  Social Sparsity! Neighborhood Systems Enrich Structured Shrinkage Operators , 2013, IEEE Transactions on Signal Processing.

[46]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[47]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[48]  Junzhou Huang,et al.  Composite splitting algorithms for convex optimization , 2011, Comput. Vis. Image Underst..

[49]  Enrico Magli,et al.  Distributed Compressed Sensing , 2015 .

[50]  Yonina C. Eldar,et al.  C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework , 2010, IEEE Transactions on Signal Processing.

[51]  P. Boesiger,et al.  SENSE: Sensitivity encoding for fast MRI , 1999, Magnetic resonance in medicine.

[52]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[53]  Minh N. Do,et al.  Tree-Based Orthogonal Matching Pursuit Algorithm for Signal Reconstruction , 2006, 2006 International Conference on Image Processing.

[54]  Michael B. Wakin,et al.  Concentration of Measure for Block Diagonal Matrices With Applications to Compressive Signal Processing , 2011, IEEE Transactions on Signal Processing.

[55]  Angshul Majumdar,et al.  Calibration-less multi-coil MR image reconstruction. , 2012, Magnetic resonance imaging.

[56]  Adolf Pfefferbaum,et al.  The SRI24 multichannel atlas of normal adult human brain structure , 2009, Human brain mapping.

[57]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[58]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[59]  L. Ying,et al.  Accelerating SENSE using compressed sensing , 2009, Magnetic resonance in medicine.

[60]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[61]  Junzhou Huang,et al.  Compressive Sensing MRI with Wavelet Tree Sparsity , 2012, NIPS.

[62]  E.J. Candes Compressive Sampling , 2022 .

[63]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[64]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[65]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[66]  Francis R. Bach,et al.  Consistency of the group Lasso and multiple kernel learning , 2007, J. Mach. Learn. Res..