A Survey on Compressive Sensing: Classical Results and Recent Advancements

Recovering sparse signals from linear measurements has demonstrated outstanding utility in a vast variety of real-world applications. Compressive sensing is the topic that studies the associated raised questions for the possibility of a successful recovery. This topic is well-nourished and numerous results are available in the literature. However, their dispersity makes it challenging and time-consuming for readers and practitioners to quickly grasp its main ideas and classical algorithms, and further touch upon the recent advancements in this surging field. Besides, the sparsity notion has already demonstrated its effectiveness in many contemporary fields. Thus, these results are useful and inspiring for further investigation of related questions in these emerging fields from new perspectives. In this survey, we gather and overview vital classical tools and algorithms in compressive sensing and describe significant recent advancements. We conclude this survey by a numerical comparison of the performance of described approaches on an interesting application.

[1]  R. Gribonval,et al.  Exact Recovery Conditions for Sparse Representations With Partial Support Information , 2013, IEEE Transactions on Information Theory.

[2]  Timothy W. Finin,et al.  SAT-based Compressive Sensing , 2019, ArXiv.

[3]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[4]  M. Fornasier,et al.  Iterative thresholding algorithms , 2008 .

[5]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[6]  Frank de Hoog,et al.  New coherence and rip analysis for weak orthogonal matching pursuit , 2014, 2014 IEEE Workshop on Statistical Signal Processing (SSP).

[7]  Nazanin Rahnavard,et al.  BCS: Compressive sensing for binary sparse signals , 2012, MILCOM 2012 - 2012 IEEE Military Communications Conference.

[8]  Qun Mo,et al.  A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit , 2015, ArXiv.

[9]  Genady Grabarnik,et al.  Sparse Modeling: Theory, Algorithms, and Applications , 2014 .

[10]  Jian Wang Support Recovery With Orthogonal Matching Pursuit in the Presence of Noise , 2015, IEEE Transactions on Signal Processing.

[11]  Zhengchun Zhou,et al.  An Optimal Condition for the Block Orthogonal Matching Pursuit Algorithm , 2018, IEEE Access.

[12]  Anru Zhang,et al.  Sharp RIP bound for sparse signal and low-rank matrix recovery , 2013 .

[13]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[14]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Tim Finin,et al.  Reinforcement Quantum Annealing: A Hybrid Quantum Learning Automata , 2020, Scientific Reports.

[16]  Ramin Ayanzadeh,et al.  Leveraging Artificial Intelligence to Advance Problem-Solving with Quantum Annealers , 2020 .

[17]  Yun-Bin Zhao,et al.  RSP-Based Analysis for Sparsest and Least $\ell_1$-Norm Solutions to Underdetermined Linear Systems , 2013, IEEE Transactions on Signal Processing.

[18]  Yonina C. Eldar,et al.  Sparsity Based Sub-wavelength Imaging with Partially Incoherent Light via Quadratic Compressed Sensing References and Links , 2022 .

[19]  Marco F. Duarte,et al.  Perfect Recovery Conditions for Non-negative Sparse Modeling , 2015, IEEE Transactions on Signal Processing.

[20]  Jinming Wen,et al.  Stable Recovery of Sparse Signals via $l_p-$Minimization , 2014, ArXiv.

[21]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[22]  Michael Möller,et al.  An adaptive inverse scale space method for compressed sensing , 2012, Math. Comput..

[23]  Jian Wang,et al.  A Sharp Condition for Exact Support Recovery With Orthogonal Matching Pursuit , 2017, IEEE Transactions on Signal Processing.

[24]  Jan Vybíral,et al.  Compressed Sensing and its Applications , 2015 .

[25]  Massimo Fornasier,et al.  Theoretical Foundations and Numerical Methods for Sparse Recovery , 2010, Radon Series on Computational and Applied Mathematics.

[26]  Mohammad Reza Mohammadi,et al.  Non-negative sparse decomposition based on constrained smoothed ℓ0 norm , 2014, Signal Process..

[27]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[28]  Farrokh Marvasti,et al.  Deterministic Construction of Binary, Bipolar, and Ternary Compressed Sensing Matrices , 2009, IEEE Transactions on Information Theory.

[29]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[30]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[31]  Jinglai Shen,et al.  Exact Support and Vector Recovery of Constrained Sparse Vectors via Constrained Matching Pursuit , 2019, 1903.07236.

[32]  Michael Elad,et al.  On the Uniqueness of Nonnegative Sparse Solutions to Underdetermined Systems of Equations , 2008, IEEE Transactions on Information Theory.

[33]  Jun Yang,et al.  A Review of Sparse Recovery Algorithms , 2019, IEEE Access.

[34]  T. Blumensath,et al.  Theory and Applications , 2011 .

[35]  Yu Wang,et al.  LightAMC: Lightweight Automatic Modulation Classification via Deep Learning and Compressive Sensing , 2020, IEEE Transactions on Vehicular Technology.

[36]  Leonid P. Yaroslavsky Is "Compressed Sensing" compressive? Can it beat the Nyquist Sampling Approach? , 2015, ArXiv.

[37]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[38]  Marc E. Pfetsch,et al.  Sparse Recovery With Integrality Constraints , 2016, Discret. Appl. Math..

[39]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[40]  Lie Wang,et al.  Shifting Inequality and Recovery of Sparse Signals , 2010, IEEE Transactions on Signal Processing.

[41]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[42]  Tommaso Melodia,et al.  Compressed-Sensing-Enabled Video Streaming for Wireless Multimedia Sensor Networks , 2012, IEEE Transactions on Mobile Computing.

[43]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[44]  Hidetoshi Nishimori,et al.  Exponential Enhancement of the Efficiency of Quantum Annealing by Non-Stoquastic Hamiltonians , 2016, Frontiers ICT.

[45]  Wengu Chen,et al.  An Optimal Recovery Condition for Sparse Signals with Partial Support Information via OMP , 2019, Circuits Syst. Signal Process..

[46]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[47]  Thong T. Do,et al.  Sparsity adaptive matching pursuit algorithm for practical compressed sensing , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[48]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[49]  John C. Mitchell,et al.  Compressive Feature Learning , 2013, NIPS.

[50]  Yun-Bin Zhao,et al.  Sparse Optimization Theory and Methods , 2018 .

[51]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[52]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[53]  Laurent Jacques,et al.  Quantized Compressive Sensing with RIP Matrices: The Benefit of Dithering , 2018, Information and Inference: A Journal of the IMA.

[54]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[55]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[56]  Yoan Shin,et al.  Deterministic Sensing Matrices in Compressive Sensing: A Survey , 2013, TheScientificWorldJournal.

[57]  Zhengchun Zhou,et al.  Deterministic Compressed Sensing Matrices From Sequences With Optimal Correlation , 2019, IEEE Access.

[58]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[59]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[60]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[61]  Yin Zhang,et al.  A Fast Algorithm for Sparse Reconstruction Based on Shrinkage, Subspace Optimization, and Continuation , 2010, SIAM J. Sci. Comput..

[62]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[63]  David L. Donoho,et al.  Precise Undersampling Theorems , 2010, Proceedings of the IEEE.

[64]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[65]  E.J. Candes Compressive Sampling , 2022 .

[66]  Hongwei Li,et al.  On recovery of block sparse signals via block generalized orthogonal matching pursuit , 2018, Signal Process..

[67]  Jian Wang,et al.  Generalized Orthogonal Matching Pursuit , 2011, IEEE Transactions on Signal Processing.

[68]  V. Kreinovich,et al.  Why ` 1 is a Good Approximation to ` 0 : A Geometric Explanation , 2012 .

[69]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[70]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[71]  David L. Donoho,et al.  Counting the Faces of Randomly-Projected Hypercubes and Orthants, with Applications , 2008, Discret. Comput. Geom..

[72]  Jean-Jacques Fuchs,et al.  Spread representations , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[73]  Ping Zhang,et al.  A secure data collection scheme based on compressive sensing in wireless sensor networks , 2018, Ad Hoc Networks.

[74]  Yanli Shi,et al.  Sparse Recovery With Block Multiple Measurement Vectors Algorithm , 2019, IEEE Access.

[75]  Marc E. Pfetsch,et al.  The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing , 2012, IEEE Transactions on Information Theory.

[76]  Shoushui Wei,et al.  Electrocardiogram Reconstruction Based on Compressed Sensing , 2019, IEEE Access.

[77]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[78]  Rick Chartrand,et al.  Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[79]  Jinglai Shen,et al.  Solution uniqueness of convex piecewise affine functions based optimization with applications to constrained ℓ1 minimization , 2017, ESAIM: Control, Optimisation and Calculus of Variations.

[80]  R. B. Deshmukh,et al.  A Systematic Review of Compressive Sensing: Concepts, Implementations and Applications , 2018, IEEE Access.

[81]  David Zhang,et al.  A Survey of Sparse Representation: Algorithms and Applications , 2015, IEEE Access.

[82]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[83]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[84]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[85]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[86]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[87]  Jun Zhang,et al.  On Recovery of Sparse Signals via ℓ1 Minimization , 2008, ArXiv.

[88]  Yue Gao,et al.  Sparse Representation for Wireless Communications: A Compressive Sensing Approach , 2018, IEEE Signal Processing Magazine.

[89]  Masaaki Nagahara,et al.  Discrete Signal Reconstruction by Sum of Absolute Values , 2015, IEEE Signal Processing Letters.

[90]  Yi Shen,et al.  A Remark on the Restricted Isometry Property in Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[91]  Feng Zhang,et al.  Sharp sufficient condition of block signal recovery via l 2/l 1-minimisation , 2017, IET Signal Process..

[92]  S. Foucart A note on guaranteed sparse recovery via ℓ1-minimization , 2010 .

[93]  Yun-Bin Zhao,et al.  Equivalence and Strong Equivalence Between the Sparsest and Least $$\ell _1$$ℓ1-Norm Nonnegative Solutions of Linear Systems and Their Applications , 2013, 1312.4163.

[94]  Stephen P. Boyd,et al.  Compressed Sensing With Quantized Measurements , 2010, IEEE Signal Processing Letters.

[95]  Yuli Fu,et al.  Block-sparse recovery via redundant block OMP , 2014, Signal Process..

[96]  Lie Wang,et al.  Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise , 2011, IEEE Transactions on Information Theory.

[97]  Hui Zhang,et al.  Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization , 2012, Journal of Optimization Theory and Applications.

[98]  Mohammad Javad Abdi,et al.  Cardinality optimization problems , 2013 .

[99]  Wotao Yin,et al.  Signal representations with minimum ℓ∞-norm , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[100]  Rayan Saab,et al.  Stable sparse approximations via nonconvex optimization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[101]  Sungyoung Lee,et al.  Compressive sensing: From theory to applications, a survey , 2013, Journal of Communications and Networks.

[102]  Massimo Fornasier,et al.  Numerical Methods for Sparse Recovery , 2010 .

[103]  Timothy W. Finin,et al.  Quantum Annealing Based Binary Compressive Sensing with Matrix Uncertainty , 2019, ArXiv.

[104]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[105]  Zongben Xu,et al.  Representative of L1/2 Regularization among Lq (0 < q ≤ 1) Regularizations: an Experimental Study Based on Phase Diagram , 2012 .

[106]  Zhengchun Zhou,et al.  A Novel Sufficient Condition for Generalized Orthogonal Matching Pursuit , 2016, IEEE Communications Letters.

[107]  Gitta Kutyniok,et al.  Theory and applications of compressed sensing , 2012, 1203.3815.

[108]  Mikhail Khodak,et al.  A Compressed Sensing View of Unsupervised Text Embeddings, Bag-of-n-Grams, and LSTMs , 2018, ICLR.

[109]  Tim Finin,et al.  An Ensemble Approach for Compressive Sensing with Quantum Annealers , 2020, IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium.

[110]  Yonina C. Eldar,et al.  GESPAR: Efficient Phase Retrieval of Sparse Signals , 2013, IEEE Transactions on Signal Processing.

[111]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[112]  Wei Dan A Sharp RIP Condition for Orthogonal Matching Pursuit , 2015 .

[113]  C. Bachoc,et al.  Applied and Computational Harmonic Analysis Tight P-fusion Frames , 2022 .

[114]  Yike Liu,et al.  High-Efficiency Observations: Compressive Sensing and Recovery of Seismic Waveform Data , 2019, Pure and Applied Geophysics.

[115]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[116]  Laurent Jacques,et al.  Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors , 2011, IEEE Transactions on Information Theory.

[117]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[118]  J. Scales,et al.  Robust methods in inverse theory , 1988 .

[119]  Song Li,et al.  New bounds on the restricted isometry constant δ2k , 2011 .

[120]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[121]  Zongben Xu,et al.  $L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast Solver , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[122]  Yonina C. Eldar,et al.  Sparse signal recovery from nonlinear measurements , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[123]  Sheng Wang,et al.  Binary Compressive Sensing via Sum of l1-Norm and l(infinity)-Norm Regularization , 2013, MILCOM 2013 - 2013 IEEE Military Communications Conference.

[124]  H. T. Kung,et al.  Stable and Efficient Representation Learning with Nonnegativity Constraints , 2014, ICML.

[125]  Naoki Abe,et al.  Grouped Orthogonal Matching Pursuit for Variable Selection and Prediction , 2009, NIPS.

[126]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[127]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[128]  Jian Wang,et al.  Multipath Matching Pursuit , 2013, IEEE Transactions on Information Theory.

[129]  Nadav Hallak,et al.  On the Minimization Over Sparse Symmetric Sets: Projections, Optimality Conditions, and Algorithms , 2016, Math. Oper. Res..

[130]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[131]  Andreas M. Tillmann Computing the spark: mixed-integer programming for the (vector) matroid girth problem , 2019, Comput. Optim. Appl..

[132]  Jinglai Shen,et al.  Least Sparsity of p-Norm Based Optimization Problems with p>1 , 2017, SIAM J. Optim..

[133]  Jinming Wen,et al.  Improved Bounds on the Restricted Isometry Constant for Orthogonal Matching Pursuit , 2013, ArXiv.

[134]  Yin Zhang,et al.  User's Guide for YALL1: Your ALgorithms for L1 Optimization , 2009 .

[135]  Gitta Kutyniok,et al.  Compressed Sensing for Finite-Valued Signals , 2016, 1609.09450.

[136]  A. Locquet,et al.  Compressive Sensing with Optical Chaos , 2016, Scientific Reports.

[137]  Yonina C. Eldar,et al.  Blind Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[138]  Bhiksha Raj,et al.  Greedy sparsity-constrained optimization , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[139]  Christian Kanzow,et al.  Mathematical Programs with Cardinality Constraints: Reformulation by Complementarity-Type Conditions and a Regularization Method , 2016, SIAM J. Optim..

[140]  Yonina C. Eldar,et al.  Sparsity Constrained Nonlinear Optimization: Optimality Conditions and Algorithms , 2012, SIAM J. Optim..

[141]  Alexandros G. Dimakis,et al.  Sparse Recovery of Nonnegative Signals With Minimal Expansion , 2011, IEEE Transactions on Signal Processing.

[142]  Lasith Adhikari Nonconvex Sparse Recovery Methods , 2017 .

[143]  Kenneth E. Barner,et al.  Iterative hard thresholding for compressed sensing with partially known support , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[144]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[145]  Alexandre d'Aspremont,et al.  Testing the nullspace property using semidefinite programming , 2008, Math. Program..

[146]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[147]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[148]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[149]  Jiming Peng,et al.  On Mehrotra-Type Predictor-Corrector Algorithms , 2007, SIAM J. Optim..

[150]  Vladik Kreinovich,et al.  Why l1 Is a Good Approximation to l0: A Geometric Explanation , 2013 .

[151]  Tianlin Liu,et al.  Binary Compressive Sensing via Smoothed $\ell_0$ Gradient Descent. , 2018 .

[152]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[153]  Rongrong Wang,et al.  Quantization of compressive samples with stable and robust recovery , 2015, ArXiv.

[154]  Tim Finin,et al.  Quantum-Assisted Greedy Algorithms , 2019 .