Greedy type algorithms for RIP matrices. A study of two selection rules

Some consequences of the Restricted Isometry Property (RIP) of matrices have been applied to develop a greedy algorithm called "ROMP" (Regularized Orthogonal Matching Pursuit) to recover sparse signals and to approximate non-sparse ones. These consequences were subsequently applied to other greedy and thresholding algorithms like "SThresh", "CoSaMP", "StOMP" and "SWCGP". In this paper, we find another consequence of the RIP property and use it to analyze the approximation to k-sparse signals with Stagewise Weak versions of Gradient Pursuit (SWGP), Matching Pursuit (SWMP) and Orthogonal Matching Pursuit (SWOMP). We combine the above mentioned algorithms with another selection rule similar to the ones that have appeared in the literature showing that results are obtained with less restrictions in the RIP constant, but we need a smaller threshold parameter for the coefficients. The results of some experiments are shown.

[1]  Pierre Vandergheynst,et al.  On the exponential convergence of matching pursuits in quasi-incoherent dictionaries , 2006, IEEE Transactions on Information Theory.

[2]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[3]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[4]  Entao Liu,et al.  Orthogonal Super Greedy Algorithm and Applications in Compressed Sensing ∗ , 2010 .

[5]  R. DeVore,et al.  Instance optimal decoding by thresholding in compressed sensing , 2008 .

[6]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[7]  Yi Shen,et al.  Remarks on the Restricted Isometry Property in Orthogonal Matching Pursuit algorithm , 2011, ArXiv.

[8]  M. Rudelson,et al.  Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[9]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[10]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[11]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[12]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[13]  Ray Maleh,et al.  Improved RIP Analysis of Orthogonal Matching Pursuit , 2011, ArXiv.

[14]  Vladimir Temlyakov,et al.  CAMBRIDGE MONOGRAPHS ON APPLIED AND COMPUTATIONAL MATHEMATICS , 2022 .

[15]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[16]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity , 2008, ArXiv.

[17]  Mike E. Davies,et al.  Stagewise Weak Gradient Pursuits , 2009, IEEE Transactions on Signal Processing.

[18]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[19]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[20]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[21]  L. Jones.  On a conjecture of Huber concerning the convergence of projection pursuit regression , 1987 .

[22]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[23]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[24]  Vladimir N. Temlyakov,et al.  Weak greedy algorithms[*]This research was supported by National Science Foundation Grant DMS 9970326 and by ONR Grant N00014‐96‐1‐1003. , 2000, Adv. Comput. Math..

[25]  Anupam Gupta,et al.  An elementary proof of the Johnson-Lindenstrauss Lemma , 1999 .

[26]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[27]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[28]  Mike E. Davies,et al.  Gradient Pursuits , 2008, IEEE Transactions on Signal Processing.

[29]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[30]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.