Introduction to Sparsity in Signal Processing 1

These notes describe how sparsity can be used in several signal processing problems. A common theme throughout these notes is the comparison between the least square solution and the sparsity-based solution. In each case, the sparsity-based solution has a clear advantage. It must be emphasized that this will not be true in general, unless the signal to be processed is either sparse or has a sparse representation with respect to a known transform. To keep the explanations as clear as possible, the examples given in these notes are restricted to 1-D signals. However, the concepts, problem formulations, and algorithms can all be used in the multidimensional case as well. For a broader overview of sparsity and compressed sensing, see the June 2010 special issue of the Proceedings of the IEEE [5], the March 2006 issue of Signal Processing [15], or the 2011 workshop SPARS11 [22].

[1]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[2]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[3]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part II: adaptive algorithms , 2006, IEEE Transactions on Image Processing.

[4]  Rémi Gribonval,et al.  Sparse approximations in signal and image processing , 2006, Signal Process..

[5]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[6]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[7]  C. Burrus,et al.  Introduction to Wavelets and Wavelet Transforms: A Primer , 1997 .

[8]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[9]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[10]  Laurent Jacques,et al.  A panorama on multiscale geometric representations, intertwining spatial, directional and frequency selectivity , 2011, Signal Process..

[11]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[12]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[13]  D. Donoho,et al.  Redundant Multiscale Transforms and Their Application for Morphological Component Separation , 2004 .

[14]  Ivan W. Selesnick,et al.  L1-norm penalized least squares with SALSA , 2014 .

[15]  André Kaup,et al.  Spatiotemporal Selective Extrapolation for 3-D Signals and Its Applications in Video Communications , 2007, IEEE Transactions on Image Processing.

[16]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[17]  Michael Elad,et al.  Applications of Sparse Representation and Compressive Sensing , 2010, Proc. IEEE.

[18]  I. Selesnick.  Sparse signal restoration , 2009 .

[19]  Michael B. Wakin.  Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity (Starck, J.-L., et al; 2010) [Book Reviews] , 2011, IEEE Signal Processing Magazine.

[20]  Antonin Chambolle,et al.  Image Decomposition into a Bounded Variation Component and an Oscillating Component , 2005, Journal of Mathematical Imaging and Vision.

[21]  Til Aach,et al.  Frequency selective signal extrapolation with applications to error concealment in image communication , 2005 .

[22]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[23]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory , 2006, IEEE Transactions on Image Processing.

[24]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[25]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[26]  Yao Wang,et al.  Error control and concealment for video communication: a review , 1998, Proc. IEEE.

[27]  Ivan W. Selesnick,et al.  Resonance-based signal decomposition: A new sparsity-enabled signal analysis method , 2011, Signal Process..

[28]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[29]  Heinz H. Bauschke,et al.  Fixed-Point Algorithms for Inverse Problems in Science and Engineering , 2011, Springer Optimization and Its Applications.

[30]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .