Random Sampling of Sparse Trigonometric Polynomials, II. Orthogonal Matching Pursuit versus Basis Pursuit

Abstract We investigate the problem of reconstructing sparse multivariate trigonometric polynomials from few randomly taken samples by Basis Pursuit and greedy algorithms such as Orthogonal Matching Pursuit (OMP) and Thresholding. While recovery by Basis Pursuit has recently been studied by several authors, we provide theoretical results on the success probability of reconstruction via Thresholding and OMP for both a continuous and a discrete probability model for the sampling points. We present numerical experiments, which indicate that usually Basis Pursuit is significantly slower than greedy algorithms, while the recovery rates are very similar.

[1]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[2]  A. I. Solomon,et al.  Extended Bell and Stirling numbers from hypergeometric exponentiation , 2001, math/0106123.

[3]  Michael Elad,et al.  On the stability of the basis pursuit in the presence of noise , 2006, Signal Process..

[4]  Holger Rauhut.  Stability Results for Random Sampling of Sparse Trigonometric Polynomials , 2008, IEEE Transactions on Information Theory.

[5]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[6]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[7]  R. Vershynin.  Frame expansions with erasures: an approach through the non-commutative operator theory , 2004, math/0405566.

[8]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[9]  H. Rauhut.  Random Sampling of Sparse Trigonometric Polynomials , 2005, math/0512642.

[10]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[11]  It Informatics.  On-Line Encyclopedia of Integer Sequences , 2010 .

[12]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[13]  Justin Romberg,et al.  Practical Signal Recovery from Random Projections , 2005 .

[14]  Joel A. Tropp,et al.  Topics in sparse approximation , 2004 .

[15]  Pierre Vandergheynst,et al.  Average Case Analysis of Multichannel Thresholding , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[16]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[17]  A. Gilbert,et al.  Theoretical and experimental analysis of a randomized algorithm for Sparse Fourier transform analysis , 2004, math/0411102.

[18]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[19]  G. Peskir.  Best constants in Kahane-Khintchine inequalities for complex Steinhaus functions , 1995 .

[20]  Pierre Vandergheynst,et al.  On the exponential convergence of matching pursuits in quasi-incoherent dictionaries , 2006, IEEE Transactions on Information Theory.

[21]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[22]  Sudipto Guha,et al.  Near-optimal sparse fourier representations via sampling , 2002, STOC '02.

[23]  M. Rudelson,et al.  Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[24]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[25]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[26]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[27]  Yishay Mansour,et al.  Randomized Interpolation and Approximation of Sparse Polynomials , 1992, SIAM J. Comput..

[28]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[29]  H. Rauhut,et al.  Learning Trigonometric Polynomials from Random Samples and Exponential Inequalities for Eigenvalues of Random Matrices , 2007, math/0701781.

[30]  J. Tropp.  On the conditioning of random subdictionaries , 2008 .

[31]  Anna C. Gilbert,et al.  Improved time bounds for near-optimal sparse Fourier representations , 2005, SPIE Optics + Photonics.

[32]  DANIEL POTTS,et al.  PROBABILITY AGAINST CONDITION NUMBER AND SAMPLING OF MULTIVARIATE TRIGONOMETRIC RANDOM POLYNOMIALS , 2006 .

[33]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[34]  Emmanuel J. Candès,et al.  Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions , 2004, Found. Comput. Math..

[35]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[36]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[37]  Jing Zou,et al.  A sublinear algorithm for the recovery of signals with sparse Fourier transform when many samples are missing , 2007 .

[38]  Gabriele Steidl,et al.  Fast Fourier Transforms for Nonequispaced Data: A Tutorial , 2001 .

[39]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[40]  G. Bennett.  Probability Inequalities for the Sum of Independent Random Variables , 1962 .

[41]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[42]  H. Halberstam,et al.  Combinatorial Analysis , 1971, Nature.

[43]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[44]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[45]  Ññøøøññøø Blockin.  Random Sampling of Multivariate Trigonometric Polynomials , 2004 .