A Study on Sparse Vector Distributions and Recovery from Compressed Sensing

I empirically investigate the variability of several recovery algorithms on the distribution underlying the sparse vector sensed by a random matrix. a dependence that has been noted before, but, to my knowledge, not thoroughly investigated. I find that ‘1-minimization [1] and tuned two-stage thresholding [2] (subspace pursuit [3] without the use of a sparsity oracle) are the most robust to changes in the sparse vector distribution; but they are outperformed to a large degree by greedy methods, such as orthogonal matching pursuit [4] for sparse vectors distributed Normal and Laplacian. I also find that selecting the best solution from those produced by several recovery algorithms can significantly increase the probability of exact recovery.

[1]  R. DeVore,et al.  Nonlinear approximation , 1998, Acta Numerica.

[2]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[3]  O. Christensen.  An introduction to frames and Riesz bases , 2002 .

[4]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[5]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[6]  Volkan Cevher,et al.  On accelerated hard thresholding methods for sparse approximation , 2011, Optical Engineering + Applications.

[7]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[8]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[9]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[10]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[11]  Bhaskar D. Rao,et al.  Performance limits of matching pursuit algorithms , 2008, 2008 IEEE International Symposium on Information Theory.

[12]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[13]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[14]  Michael Elad,et al.  A Plurality of Sparse Representations Is Better Than the Sparsest One Alone , 2009, IEEE Transactions on Information Theory.

[15]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[16]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[17]  JuttenChristian,et al.  A fast approach for overcomplete sparse decomposition based on smoothed l0 norm , 2009 .

[18]  Arian Maleki,et al.  Optimally Tuned Iterative Reconstruction Algorithms for Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[19]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[20]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[21]  S. Jensen,et al.  The Cyclic Matching Pursuit and its Application to Audio Modeling and Coding , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[22]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[23]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[24]  Piotr J. Durka,et al.  Stochastic time-frequency dictionaries for matching pursuit , 2001, IEEE Trans. Signal Process..

[25]  Jian Li,et al.  Multi-model approach to model selection , 2004, Digit. Signal Process..

[26]  Okan K. Ersoy,et al.  Probabilistic Matching Pursuit for Compressive Sensing , 2010 .

[27]  Aleksandar Dogandzic,et al.  Variance-Component Based Sparse Signal Reconstruction and Model Selection , 2010, IEEE Transactions on Signal Processing.

[28]  Jared Tanner,et al.  Phase Transitions for Greedy Sparse Approximation Algorithms , 2010, ArXiv.

[29]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[30]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[31]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[32]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[33]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[34]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.