Orthogonal Matching Pursuit: A Brownian Motion Analysis

A well-known analysis of Tropp and Gilbert shows that orthogonal matching pursuit (OMP) can recover a <i>k</i>-sparse <i>n</i>-dimensional real vector from <i>m</i>=4<i>k</i>log(<i>n</i>) noise-free linear measurements obtained through a random Gaussian measurement matrix with a probability that approaches one as <i>n</i>→∞. This work strengthens this result by showing that a lower number of measurements, <i>m</i>=2<i>k</i>log(<i>n</i>-<i>k</i>) , is in fact sufficient for asymptotic recovery. More generally, when the sparsity level satisfies <i>k</i><sub>min</sub> ≤ <i>k</i> ≤ <i>k</i><sub>max</sub> but is unknown, <i>m</i>=2<i>k</i><sub>max</sub>log(<i>n</i>-<i>k</i><sub>min</sub>) measurements is sufficient. Furthermore, this number of measurements is also sufficient for detection of the sparsity pattern (support) of the vector with measurement errors provided the signal-to-noise ratio (SNR) scales to infinity. The scaling <i>m</i>=2<i>k</i>log(<i>n</i>-<i>k</i>) exactly matches the number of measurements required by the more complex lasso method for signal recovery with a similar SNR scaling.

[1]  K. B. Oldham,et al.  An Atlas of Functions. , 1988 .

[2]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[3]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[4]  S. Rangan,et al.  Orthogonal Matching Pursuit from Noisy Measurements : A New Analysis ∗ , 2009 .

[5]  Sundeep Rangan,et al.  On-Off Random Access Channels: A Compressed Sensing Framework , 2009, ArXiv.

[6]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[7]  Sundeep Rangan,et al.  Ranked Sparse Signal Support Detection , 2011, IEEE Transactions on Signal Processing.

[8]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing, 2nd Edition , 1999 .

[9]  Sundeep Rangan,et al.  A sparsity detection framework for on-off random access channels , 2009, 2009 IEEE International Symposium on Information Theory.

[10]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[11]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[12]  Bhaskar D. Rao,et al.  Comparing the Effects of Different Weight Distributions on Finding Sparse Representations , 2005, NIPS.

[13]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[14]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[15]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[16]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.

[17]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[18]  Venkatesh Saligrama,et al.  Information Theoretic Bounds for Compressed Sensing , 2008, IEEE Transactions on Information Theory.

[19]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[20]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[21]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[22]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[23]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[24]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[25]  Martin J. Wainwright,et al.  Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting , 2009, IEEE Trans. Inf. Theory.

[26]  Bhaskar D. Rao,et al.  Performance limits of matching pursuit algorithms , 2008, 2008 IEEE International Symposium on Information Theory.

[27]  Xiongzhi Chen.  Brownian Motion and Stochastic Calculus , 2008 .

[28]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[29]  G. Reeves.  Sparse Signal Sampling using Noisy Linear Projections , 2008 .

[30]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[31]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[32]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[33]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[34]  Vahid Tarokh,et al.  Shannon-Theoretic Limits on Noisy Compressive Sampling , 2007, IEEE Transactions on Information Theory.

[35]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[36]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[37]  Sundeep Rangan,et al.  Orthogonal Matching Pursuit From Noisy Random Measurements: A New Analysis , 2009, NIPS.

[38]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[39]  Sundeep Rangan,et al.  Necessary and Sufficient Conditions for Sparsity Pattern Recovery , 2008, IEEE Transactions on Information Theory.

[40]  A. Atkinson.  Subset Selection in Regression , 1992 .

[41]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.