A Survey of Compressed Sensing

Compressed sensing was introduced some ten years ago as an effective way of acquiring signals, which possess a sparse or nearly sparse representation in a suitable basis or dictionary. Due to its solid mathematical backgrounds, it quickly attracted the attention of mathematicians from several different areas, so that the most important aspects of the theory are nowadays very well understood. In recent years, its applications started to spread out through applied mathematics, signal processing, and electrical engineering. The aim of this chapter is to provide an introduction into the basic concepts of compressed sensing. In the first part of this chapter, we present the basic mathematical concepts of compressed sensing, including the Null Space Property, Restricted Isometry Property, their connection to basis pursuit and sparse recovery, and construction of matrices with small restricted isometry constants. This presentation is easily accessible, largely self-contained, and includes proofs of the most important theorems. The second part gives an overview of the most important extensions of these ideas, including recovery of vectors with sparse representation in frames and dictionaries, discussion of (in)coherence and its implications for compressed sensing, and presentation of other algorithms of sparse recovery.

[1]  D. Du,et al.  Combinatorial Group Testing and Its Applications , 1993 .

[2]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[3]  R. Vershynin,et al.  One sketch for all: fast algorithms for compressed sensing , 2007, STOC '07.

[4]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[5]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[6]  Dustin G. Mixon,et al.  Kirkman Equiangular Tight Frames and Codes , 2013, IEEE Transactions on Information Theory.

[7]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[8]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[9]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[10]  R. Dorfman The Detection of Defective Members of Large Populations , 1943 .

[11]  M. Talagrand,et al.  Probability in Banach spaces , 1991 .

[12]  I. Loris On the performance of algorithms for the minimization of ℓ1-penalized functionals , 2007, 0710.4082.

[13]  I. Daubechies,et al.  Capturing Ridge Functions in High Dimensions from Point Queries , 2012 .

[14]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[15]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[16]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[17]  R. DeVore,et al.  Approximation of Functions of Few Variables in High Dimensions , 2011 .

[18]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[19]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[20]  H. Rauhut Random Sampling of Sparse Trigonometric Polynomials , 2005, math/0512642.

[21]  R. Baraniuk,et al.  Compressive Radar Imaging , 2007, 2007 IEEE Radar Conference.

[22]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[23]  Lie Wang,et al.  Shifting Inequality and Recovery of Sparse Signals , 2010, IEEE Transactions on Signal Processing.

[24]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[25]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[26]  Bernd Gärtner,et al.  Understanding and using linear programming , 2007, Universitext.

[27]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[28]  Rachel Ward,et al.  New and Improved Johnson-Lindenstrauss Embeddings via the Restricted Isometry Property , 2010, SIAM J. Math. Anal..

[29]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[30]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[31]  S. Foucart A note on guaranteed sparse recovery via ℓ1-minimization , 2010 .

[32]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[33]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[34]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[35]  D. L. Donoho,et al.  Compressed sensing , 2006, IEEE Trans. Inf. Theory.

[36]  Pierre Vandergheynst,et al.  Compressed Sensing and Redundant Dictionaries , 2007, IEEE Transactions on Information Theory.

[37]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[38]  慧 廣瀬 A Mathematical Introduction to Compressive Sensing , 2015 .

[39]  Lloyd R. Welch,et al.  Lower bounds on the maximum cross correlation of signals (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[40]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[41]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[42]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[43]  Richard G. Baraniuk,et al.  Wavelet-domain compressive signal reconstruction using a Hidden Markov Tree model , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Babak Hassibi,et al.  On the reconstruction of block-sparse signals with an optimal number of measurements , 2009, IEEE Trans. Signal Process..

[45]  P. Wojtaszczyk,et al.  Complexity of approximation of functions of few variables in high dimensions , 2011, J. Complex..

[46]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[47]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[48]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[49]  Richard G. Baraniuk,et al.  Recovery of Jointly Sparse Signals from Few Random Projections , 2005, NIPS.

[50]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[51]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[52]  J. Matousek,et al.  On variants of the Johnson–Lindenstrauss lemma , 2008 .

[53]  Volkan Cevher,et al.  Active Learning of Multi-Index Function Models , 2012, NIPS.

[54]  Jan Vybíral,et al.  Learning Functions of Few Arbitrary Linear Parameters in High Dimensions , 2010, Found. Comput. Math..

[55]  Yonina C. Eldar,et al.  Introduction to Compressed Sensing , 2022 .

[56]  Yonina C. Eldar,et al.  From Theory to Practice: Sub-Nyquist Sampling of Sparse Wideband Analog Signals , 2009, IEEE Journal of Selected Topics in Signal Processing.

[57]  Ely Porat,et al.  Approximate sparse recovery: optimizing time and measurements , 2009, STOC '10.

[58]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[59]  M. Ledoux The concentration of measure phenomenon , 2001 .

[60]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[61]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[62]  Holger Rauhut,et al.  Sparse Legendre expansions via l1-minimization , 2012, J. Approx. Theory.

[63]  Jan Vybíral,et al.  Compressed learning of high-dimensional sparse functions , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[64]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[65]  Richard G. Baraniuk,et al.  Fast reconstruction of piecewise smooth signals from random projections , 2005 .

[66]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[67]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[68]  Minh N. Do,et al.  Tree-Based Orthogonal Matching Pursuit Algorithm for Signal Reconstruction , 2006, 2006 International Conference on Image Processing.

[69]  V. Milman,et al.  Asymptotic Theory Of Finite Dimensional Normed Spaces , 1986 .

[70]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[71]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[72]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[73]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[74]  Lie Wang,et al.  New Bounds for Restricted Isometry Constants , 2009, IEEE Transactions on Information Theory.

[75]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.