Model-Based Compressive Sensing

Compressive sensing (CS) is an alternative to Shannon/Nyquist sampling for the acquisition of sparse or compressible signals that can be well approximated by just K ¿ N elements from an N -dimensional basis. Instead of taking periodic samples, CS measures inner products with M < N random vectors and then recovers the signal via a sparsity-seeking optimization or greedy algorithm. Standard CS dictates that robust signal recovery is possible from M = O(K log(N/K)) measurements. It is possible to substantially decrease M without sacrificing robustness by leveraging more realistic signal models that go beyond simple sparsity and compressibility by including structural dependencies between the values and locations of the signal coefficients. This paper introduces a model-based CS theory that parallels the conventional theory and provides concrete guidelines on how to create model-based recovery algorithms with provable performance guarantees. A highlight is the introduction of a new class of structured compressible signals along with a new sufficient condition for robust structured compressible signal recovery that we dub the restricted amplification property, which is the natural counterpart to the restricted isometry property of conventional CS. Two examples integrate two relevant signal models-wavelet trees and block sparsity-into two state-of-the-art CS recovery algorithms and prove that they offer robust recovery from just M = O(K) measurements. Extensive numerical simulations demonstrate the validity and applicability of our new theory and algorithms.

[1]  Gerald G. Brown,et al.  On Random Binary Trees , 1984, Math. Oper. Res..

[2]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[3]  Douglas L. Jones,et al.  A signal-dependent time-frequency representation: fast algorithm for optimal kernel design , 1994, IEEE Trans. Signal Process..

[4]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[5]  D. Donoho CART AND BEST-ORTHO-BASIS: A CONNECTION' , 1997 .

[6]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[7]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[8]  S. Mallat A wavelet tour of signal processing , 1998 .

[9]  Richard G. Baraniuk,et al.  Optimal tree approximation with wavelets , 1999, Optics & Photonics.

[10]  M. Ledoux The concentration of measure phenomenon , 2001 .

[11]  I. Daubechies,et al.  Tree Approximation and Optimal Encoding , 2001 .

[12]  Justin K. Romberg,et al.  Bayesian tree-structured image modeling using wavelet-domain hidden Markov models , 2001, IEEE Trans. Image Process..

[13]  Richard G. Baraniuk,et al.  Near Best Tree Approximation , 2002, Adv. Comput. Math..

[14]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[15]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[16]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[17]  Richard G. Baraniuk,et al.  Fast reconstruction of piecewise smooth signals from random projections , 2005 .

[18]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[19]  Emmanuel J. Candès,et al.  Signal recovery from random projections , 2005, IS&T/SPIE Electronic Imaging.

[20]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[21]  Richard G. Baraniuk,et al.  Recovery of Jointly Sparse Signals from Few Random Projections , 2005, NIPS.

[22]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[23]  Minh N. Do,et al.  Tree-Based Orthogonal Matching Pursuit Algorithm for Signal Reconstruction , 2006, 2006 International Conference on Image Processing.

[24]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[25]  E.J. Candes Compressive Sampling , 2022 .

[26]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[27]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[28]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[29]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[30]  M. Do,et al.  Tree-Based Majorize-Maximize Algorithm for Compressed Sensing with Sparse-Tree Prior , 2007, 2007 2nd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing.

[31]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[32]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[33]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[34]  Volkan Cevher,et al.  Sparse Signal Recovery Using Markov Random Fields , 2008, NIPS.

[35]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[36]  Yue M. Lu,et al.  Sampling Signals from a Union of Subspaces , 2008, IEEE Signal Processing Magazine.

[37]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity , 2008, ArXiv.

[38]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[39]  Richard G. Baraniuk,et al.  Wavelet-domain compressive signal reconstruction using a Hidden Markov Tree model , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[40]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Union of Subspaces , 2008, ArXiv.

[41]  T. Blumensath,et al.  Sampling Theorems for Signals from the Union of Linear Subspaces , 2008 .

[42]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[43]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[44]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[45]  Richard G. Baraniuk,et al.  Distributed Compressive Sensing , 2009, ArXiv.

[46]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[47]  Volkan Cevher,et al.  Model-based compressive sensing for signal ensembles , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[48]  Marco F. Duarte,et al.  Compressive sensing recovery of spike trains using a structured sparsity model , 2009 .

[49]  Richard Baraniuk,et al.  Recovery of Clustered Sparse Signals from Compressive Measurements , 2009 .

[50]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[51]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.