A Tutorial on Sparse Signal Reconstruction and Its Applications in Signal Processing

Sparse signals are characterized by a few nonzero coefficients in one of their transformation domains. This was the main premise in designing signal compression algorithms. Compressive sensing as a new approach employs the sparsity property as a precondition for signal recovery. Sparse signals can be fully reconstructed from a reduced set of available measurements. The description and basic definitions of sparse signals, along with the conditions for their reconstruction, are discussed in the first part of this paper. The numerous algorithms developed for the sparse signals reconstruction are divided into three classes. The first one is based on the principle of matching components. Analysis of noise and nonsparsity influence on reconstruction performance is provided. The second class of reconstruction algorithms is based on the constrained convex form of problem formulation where linear programming and regression methods can be used to find a solution. The third class of recovery algorithms is based on the Bayesian approach. Applications of the considered approaches are demonstrated through various illustrative and signal processing examples, using common transformation and observation matrices. With pseudocodes of the presented algorithms and compressive sensing principles illustrated on simple signal processing examples, this tutorial provides an inductive way through this complex field to researchers and practitioners starting from the basics of sparse signal processing up to the most recent and up-to-date methods and signal processing applications.

[1]  Jean-Luc Starck,et al.  Compressed Sensing in Astronomy , 2008, IEEE Journal of Selected Topics in Signal Processing.

[2]  Jung-San Lee,et al.  Selective scalable secret image sharing with verification , 2015, Multimedia Tools and Applications.

[3]  Ljubiša Stanković,et al.  Adaptive Variable Step Algorithm for Missing Samples Recovery in Sparse Signals , 2013, IET Signal Process..

[4]  M. Fornasier,et al.  Iterative thresholding algorithms , 2008 .

[5]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[6]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[7]  Irena Orovic,et al.  Relationship between the robust statistics theory and sparse compressive sensed signals reconstruction , 2014, IET Signal Process..

[8]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[9]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[10]  Irena Orovic,et al.  Performance of Compressive Sensing Image Reconstruction for Search and Rescue , 2016, IEEE Geoscience and Remote Sensing Letters.

[11]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[12]  Sungyoung Lee,et al.  Compressive sensing: From theory to applications, a survey , 2013, Journal of Communications and Networks.

[13]  Irena Orovic,et al.  Improved higher order robust distributions based on compressive sensing reconstruction , 2014, IET Signal Process..

[14]  Ljubisa Stankovic,et al.  Nonsparsity influence on the ISAR recovery from reduced data [Correspondence] , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[15]  Bhaskar D. Rao,et al.  Extension of SBL Algorithms for the Recovery of Block Sparse Signals With Intra-Block Correlation , 2012, IEEE Transactions on Signal Processing.

[16]  Georgios B. Giannakis,et al.  Distributed Spectrum Sensing for Cognitive Radio Networks by Exploiting Sparsity , 2010, IEEE Transactions on Signal Processing.

[17]  G. Giannakis,et al.  Compressed sensing of time-varying signals , 2009, 2009 16th International Conference on Digital Signal Processing.

[18]  Irena Orovic,et al.  L-statistics based modification of reconstruction algorithms for compressive sensing in the presence of impulse noise , 2013, Signal Process..

[19]  Fathi E. Abd El-Samie,et al.  Compressive sensing applied to radar systems: an overview , 2015, Signal Image Video Process..

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Moeness G. Amin,et al.  Compressive sensing for urban radar , 2014 .

[22]  Di Xiao,et al.  Separable data hiding in encrypted image based on compressive sensing , 2014 .

[23]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[24]  Sabine Van Huffel,et al.  Compressed Sensing of Multichannel EEG Signals: The Simultaneous Cosparsity and Low-Rank Optimization , 2015, IEEE Transactions on Biomedical Engineering.

[25]  LJubisa Stankovic,et al.  A measure of some time-frequency distributions concentration , 2001, Signal Process..

[26]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[27]  Ljubisa Stankovic,et al.  Reconstruction of Sparse Signals in Impulsive Disturbance Environments , 2017, Circuits Syst. Signal Process..

[28]  Bhaskar D. Rao,et al.  Sparse Signal Recovery With Temporally Correlated Source Vectors Using Sparse Bayesian Learning , 2011, IEEE Journal of Selected Topics in Signal Processing.

[29]  Shengli Zhou,et al.  Application of compressive sensing to sparse channel estimation , 2010, IEEE Communications Magazine.

[30]  Michael Elad,et al.  Multi-Scale Dictionary Learning Using Wavelets , 2011, IEEE Journal of Selected Topics in Signal Processing.

[31]  Lu Wang,et al.  Hierarchical Sparse Signal Recovery by Variational Bayesian Inference , 2014, IEEE Signal Processing Letters.

[32]  Jelena Kovacevic,et al.  Compression of QRS complexes using Hermite expansion , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[34]  Mike E. Davies,et al.  Gradient Pursuits , 2008, IEEE Transactions on Signal Processing.

[35]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[36]  Yimin Zhang,et al.  Detection of weak astronomical signals with frequency-hopping interference suppression , 2018, Digit. Signal Process..

[37]  Guoan Bi,et al.  Time-frequency representation reconstruction based on the compressive sensing , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[38]  Aleksandra Pizurica,et al.  Compressed sensing in MRI with a Markov random field prior for spatial clustering of subband coefficients , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[39]  Joachim H. G. Ender,et al.  On compressive sensing applied to radar , 2010, Signal Process..

[40]  Rama Chellappa,et al.  Sparse Representations and Compressive Sensing for Imaging and Vision , 2013, Springer Briefs in Electrical and Computer Engineering.

[41]  LJubisa Stankovic,et al.  ISAR image analysis and recovery with unavailable or heavily corrupted data , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[42]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[43]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[44]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[45]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[46]  N. Ahmed,et al.  Discrete Cosine Transform , 1996 .

[47]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[48]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[49]  Guoan Bi,et al.  The Group Lasso for Stable Recovery of Block-Sparse Signal Representations , 2011, IEEE Transactions on Signal Processing.

[50]  Irena Orovic,et al.  On the parameterization of Hermite transform with application to the compression of QRS complexes , 2017, Signal Process..

[51]  Moeness G. Amin,et al.  Structure-aware Bayesian compressive sensing for frequency-hopping spectrum estimation , 2016, SPIE Commercial + Scientific Sensing and Imaging.

[52]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[53]  Irena Orovic,et al.  Gradient Compressive Sensing for Image Data Reduction in UAV Based Search and Rescue in the Wild , 2016 .

[54]  Ljubiša Stanković,et al.  Analysis of the Reconstruction of Sparse Signals in the DCT Domain Applied to Audio Signals , 2018, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[55]  Irena Orovic,et al.  Compressive Sensing Approach in the Hermite Transform Domain , 2015 .

[56]  E. Candes,et al.  11-magic : Recovery of sparse signals via convex programming , 2005 .

[57]  Marco Donald Migliore,et al.  Compressed Sensing: Applications in Radar and Communications , 2016, TheScientificWorldJournal.

[58]  Wei Zhang,et al.  ISAR Imaging by Two-Dimensional Convex Optimization-Based Compressive Sensing , 2016, IEEE Sensors Journal.

[59]  Irena Orovic,et al.  An Approach to 2D Signals Recovering in Compressive Sensing Context , 2017, Circuits Syst. Signal Process..

[60]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[61]  Milos Dakovic,et al.  On the reconstruction of nonsparse time-frequency signals with sparsity constraint from a reduced set of samples , 2018, Signal Process..

[62]  Jiabin Jia,et al.  Image reconstruction algorithm for electrical impedance tomography based on block sparse Bayesian learning , 2017, 2017 IEEE International Conference on Imaging Systems and Techniques (IST).

[63]  Tzyy-Ping Jung,et al.  Compressed Sensing of EEG for Wireless Telemonitoring With Low Energy Consumption and Inexpensive Hardware , 2012, IEEE Transactions on Biomedical Engineering.

[64]  Irena Orovic,et al.  Separation and reconstruction of the rigid body and micro-Doppler signal in ISAR part II – statistical analysis , 2015 .

[65]  Shuilong He,et al.  Compressed sparse time–frequency feature representation via compressive sensing and its applications in fault diagnosis , 2015 .

[66]  Irena Orovic,et al.  Denoising of sparse images in impulsive disturbance environment , 2018, Multimedia Tools and Applications.

[67]  Irena Orovic,et al.  Multimedia Signals and Systems: Basic and Advanced Algorithms for Signal Processing , 2015 .

[68]  Stephen A. Martucci,et al.  Symmetric convolution and the discrete sine and cosine transforms , 1993, IEEE Trans. Signal Process..

[69]  Emre Ertin,et al.  Sparsity and Compressed Sensing in Radar Imaging , 2010, Proceedings of the IEEE.

[70]  Irena Orovic,et al.  An automated signal reconstruction method based on analysis of compressive sensed signals in noisy environment , 2014, Signal Process..

[71]  Irena Orovic,et al.  Video Frames Reconstruction Based on Time-Frequency Analysis and Hermite Projection Method , 2010, EURASIP J. Adv. Signal Process..

[72]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[73]  Yonina C. Eldar,et al.  Noise Folding in Compressed Sensing , 2011, IEEE Signal Processing Letters.

[74]  Aswin C. Sankaranarayanan,et al.  Compressive Video Sensing: Algorithms, architectures, and applications , 2017, IEEE Signal Processing Magazine.

[75]  Mengdao Xing,et al.  Achieving Higher Resolution ISAR Imaging With Limited Pulses via Compressed Sampling , 2009, IEEE Geoscience and Remote Sensing Letters.

[76]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[77]  Jianqin Zhou,et al.  On discrete cosine transform , 2011, ArXiv.

[78]  T. Blumensath,et al.  Theory and Applications , 2011 .

[79]  James H. McClellan,et al.  Sparse-promoting Full Waveform Inversion based on Online Orthonormal Dictionary Learning , 2015, ArXiv.

[80]  G. Clifford,et al.  A Review of Fetal ECG Signal Processing; Issues and Promising Directions. , 2010, The open pacing, electrophysiology & therapy journal.

[81]  David Cooper,et al.  Improved Compressed Sensing-Based Algorithm for Sparse-View CT Image Reconstruction , 2013, Comput. Math. Methods Medicine.

[82]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[83]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[84]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[85]  Cornel Ioana,et al.  Error in the Reconstruction of Nonsparse Images , 2018 .

[86]  Cornel Ioana,et al.  Compressive Sensing in Signal Processing: Algorithms and Transform Domain Formulations , 2016 .

[87]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[88]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[89]  Yong Xiang,et al.  When Compressive Sensing Meets Data Hiding , 2016, IEEE Signal Processing Letters.

[90]  Ljubisa Stankovic,et al.  On the Uniqueness of the Sparse Signals Reconstruction Based on the Missing Samples Variation Analysis , 2015 .

[91]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[92]  S. Luo,et al.  A review of electrocardiogram filtering. , 2010, Journal of electrocardiology.

[93]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[94]  Irena Orovic,et al.  Polynomial Fourier domain as a domain of signal sparsity , 2014, Signal Process..

[95]  Jelena Kovacevic,et al.  Efficient Compression of QRS Complexes Using Hermite Expansion , 2012, IEEE Transactions on Signal Processing.

[96]  Irena Orovic,et al.  Compressive Sensing Based Separation of Nonstationary and Stationary Signals Overlapping in Time-Frequency , 2013, IEEE Transactions on Signal Processing.

[97]  L. Breiman.  Better subset regression using the nonnegative garrote , 1995 .

[98]  P Lander,et al.  Principles and signal processing techniques of the high-resolution electrocardiogram. , 1992, Progress in cardiovascular diseases.

[99]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[100]  Irena Orovic,et al.  Separation and reconstruction of the rigid body and micro-Doppler signal in ISAR part I – theory , 2015 .

[101]  Caner Özdemii̇r,et al.  Inverse Synthetic Aperture Radar Imaging With MATLAB Algorithms , 2012 .

[102]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[103]  Graeme Pope,et al.  Compressive sensing: a summary of reconstruction algorithms , 2009 .

[104]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[105]  Irena Orovic,et al.  Time-Frequency Analysis and Hermite Projection Method Applied to Swallowing Accelerometry Signals , 2010, EURASIP J. Adv. Signal Process..

[106]  Xin Liao,et al.  Separable data hiding in encrypted image based on compressive sensing and discrete fourier transform , 2017, Multimedia Tools and Applications.

[107]  Dustin G. Mixon,et al.  Certifying the Restricted Isometry Property is Hard , 2012, IEEE Transactions on Information Theory.

[108]  Patrick Flandrin,et al.  Time-Frequency Energy Distributions Meet Compressed Sensing , 2010, IEEE Transactions on Signal Processing.

[109]  Caner Ozdemir,et al.  Inverse Synthetic Aperture Radar Imaging with MATLAB® Algorithms , 2012 .

[110]  Wei Lu,et al.  Regularized Modified BPDN for Noisy Sparse Reconstruction With Partial Erroneous Support and Signal Value Knowledge , 2010, IEEE Transactions on Signal Processing.

[111]  Jean-Bernard Martens,et al.  The Hermite transform-theory , 1990, IEEE Trans. Acoust. Speech Signal Process..

[112]  Nasser Eslahi,et al.  Compressive Sensing Image Restoration Using Adaptive Curvelet Thresholding and Nonlocal Sparse Regularization , 2016, IEEE Transactions on Image Processing.

[113]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[114]  Liam Kilmartin,et al.  Compressed Sensing for Bioelectric Signals: A Review , 2015, IEEE Journal of Biomedical and Health Informatics.

[115]  Srdjan Stankovic,et al.  Missing samples analysis in signals for applications to L-estimation and compressive sensing , 2014, Signal Process..

[116]  Jared Tanner,et al.  Explorer Compressed Sensing : How Sharp Is the Restricted Isometry Property ? , 2011 .

[117]  Irena Orovic,et al.  Time-frequency-based instantaneous frequency estimation of sparse signals from incomplete set of samples , 2014, IET Signal Process..

[118]  Ljubisa Stankovic,et al.  Time-Frequency Signal Analysis With Applications , 2013 .

[119]  Irena Orovic,et al.  Gradient-based signal reconstruction algorithm in Hermite transform domain , 2016 .

[120]  Aleksandra Pizurica,et al.  Compressed sensing using sparse adaptive measurements , 2014 .

[121]  Ran Tao,et al.  Structure-Aware Bayesian Compressive Sensing for Frequency-Hopping Spectrum Estimation With Missing Observations , 2016, IEEE Transactions on Signal Processing.

[122]  Isidora Stanković,et al.  On the Errors in Randomly Sampled Nonsparse Signals Reconstructed With a Sparsity Assumption , 2017, IEEE Geoscience and Remote Sensing Letters.