Necessary and Sufficient Conditions for Sparsity Pattern Recovery

The paper considers the problem of detecting the sparsity pattern of a k -sparse vector in \BBR n from m random noisy measurements. A new necessary condition on the number of measurements for asymptotically reliable detection with maximum-likelihood (ML) estimation and Gaussian measurement matrices is derived. This necessary condition for ML detection is compared against a sufficient condition for simple maximum correlation (MC) or thresholding algorithms. The analysis shows that the gap between thresholding and ML can be described by a simple expression in terms of the total signal-to-noise ratio (SNR), with the gap growing with increasing SNR. Thresholding is also compared against the more sophisticated Lasso and orthogonal matching pursuit (OMP) methods. At high SNRs, it is shown that the gap between Lasso and OMP over thresholding is described by the range of powers of the nonzero component values of the unknown signals. Specifically, the key benefit of Lasso and OMP over thresholding is the ability of Lasso and OMP to detect signals with relatively small components.

[1]  Vahid Tarokh,et al.  Noisy compressive sampling limits in linear and sublinear regimes , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[2]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[3]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[4]  Vivek K Goyal,et al.  Rate-Distortion Bounds for Sparse Approximation , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[5]  Pierre Vandergheynst,et al.  Compressed Sensing and Redundant Dictionaries , 2007, IEEE Transactions on Information Theory.

[6]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[7]  Vahid Tarokh,et al.  Shannon-Theoretic Limits on Noisy Compressive Sampling , 2007, IEEE Transactions on Information Theory.

[8]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  Sundeep Rangan,et al.  A sparsity detection framework for on-off random access channels , 2009, 2009 IEEE International Symposium on Information Theory.

[10]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[11]  Venkatesh Saligrama,et al.  Fundamental Limits on Sensing Capacity for Sensor Networks and Compressed Sensing , 2008, ArXiv.

[12]  Vivek K. Goyal,et al.  Sparse approximation, denoising, and large random frames , 2005, SPIE Optics + Photonics.

[13]  Sundeep Rangan,et al.  Orthogonal Matching Pursuit From Noisy Random Measurements: A New Analysis , 2009, NIPS.

[14]  V.K. Goyal,et al.  Compressive Sampling and Lossy Compression , 2008, IEEE Signal Processing Magazine.

[15]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[16]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[17]  K. B. Oldham,et al.  An Atlas of Functions. , 1988 .

[18]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[19]  Sundeep Rangan,et al.  Resolution Limits of Sparse Coding in High Dimensions , 2008, NIPS.

[20]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[21]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[22]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[23]  G. Reeves.  Sparse Signal Sampling using Noisy Linear Projections , 2008 .

[24]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[25]  Venkatesh Saligrama,et al.  On sensing capacity of sensor networks for the class of linear observation, fixed SNR models , 2007, ArXiv.

[26]  Kannan Ramchandran,et al.  Denoising by Sparse Approximation: Error Bounds Based on Rate-Distortion Theory , 2006, EURASIP J. Adv. Signal Process..

[27]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[28]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[29]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[30]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[31]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[32]  A. Atkinson.  Subset Selection in Regression , 1992 .

[33]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices , 2008, IEEE Transactions on Information Theory.

[34]  Richard G. Baraniuk,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[35]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[36]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[37]  Sundeep Rangan,et al.  On-Off Random Access Channels: A Compressed Sensing Framework , 2009, ArXiv.

[38]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[39]  R. E. Wheeler.  Statistical distributions , 1983, APLQ.

[40]  M. Evans.  Statistical Distributions , 2000 .

[41]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[42]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[43]  Martin J. Wainwright,et al.  Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting , 2009, IEEE Trans. Inf. Theory.