Enhancing the fundamental limits of sparsity pattern recovery

Abstract Detecting the sparsity pattern or support set of a sparse vector from a small number of noisy linear measurements is a challenging problem in compressed sensing. This paper considers the problem of support recovery when statistical side information is available. From the standard linear and noisy measurement model with arbitrary sensing matrix and Gaussian additive noise and by exploiting the side information, a new linear model is derived which benefits from a larger sample size. The common potential benefits of the increase in the number of samples are revealed. The stability guarantees are then analyzed based on the new model. Two decoding schemes are taken for the support recovery task from the new framework, namely, Maximum Likelihood (ML) and Joint-Typicality (JT) decoding. Performance bounds of the support recovery from the new framework are developed and upper bounds are derived on the error probability of these decoders when they are fed with the prior knowledge which is the statistical properties of the new measurement noise. Finally, an extension is provided for when the noise is non-Gaussian. The results show that with the aid of the prior knowledge and using the new framework one can push the performance limits of the sparsity pattern recovery significantly. The approach is supported by extensive simulations including extension of LASSO to the new framework.

[1]  Alan J. Miller Subset Selection in Regression , 1992 .

[2]  P. P. Vaidyanathan,et al.  Sparse Sensing With Co-Prime Samplers and Arrays , 2011, IEEE Transactions on Signal Processing.

[3]  Martin J. Wainwright,et al.  Information-theoretic bounds on model selection for Gaussian Markov random fields , 2010, 2010 IEEE International Symposium on Information Theory.

[4]  P. P. Vaidyanathan,et al.  Pushing the Limits of Sparse Support Recovery Using Correlation Information , 2015, IEEE Transactions on Signal Processing.

[5]  John C. Duchi,et al.  The Generalization Ability of Online Algorithms for Dependent Data , 2011, IEEE Transactions on Information Theory.

[6]  Jamie S. Evans,et al.  Compressed Sensing With Prior Information: Information-Theoretic Limits and Practical Decoders , 2013, IEEE Transactions on Signal Processing.

[7]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[8]  Jarvis D. Haupt,et al.  On the Fundamental Limits of Recovering Tree Sparse Vectors From Noisy Linear Measurements , 2013, IEEE Transactions on Information Theory.

[9]  Luis Alonso,et al.  Model based compressed sensing reconstruction algorithms for ECG telemonitoring in WBANs , 2014, Digit. Signal Process..

[10]  Martin J. Wainwright,et al.  Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting , 2009, IEEE Trans. Inf. Theory.

[11]  Tareq Y. Al-Naffouri,et al.  Distribution agnostic structured sparsity recovery algorithms , 2013, 2013 8th International Workshop on Systems, Signal Processing and their Applications (WoSSPA).

[12]  Shlomo Shamai,et al.  Support Recovery With Sparsely Sampled Free Random Matrices , 2011, IEEE Transactions on Information Theory.

[13]  Yihong Wu,et al.  Rényi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression , 2010, IEEE Transactions on Information Theory.

[14]  Bhaskar D. Rao,et al.  Limits on Support Recovery of Sparse Signals via Multiple-Access Communication Techniques , 2011, IEEE Transactions on Information Theory.

[15]  Venkatesh Saligrama,et al.  Information Theoretic Bounds for Compressed Sensing , 2008, IEEE Transactions on Information Theory.

[16]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[17]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices , 2008, IEEE Transactions on Information Theory.

[18]  Richard M. Leahy,et al.  Electromagnetic brain mapping , 2001, IEEE Signal Process. Mag..

[19]  Galen Reeves,et al.  Approximate Sparsity Pattern Recovery: Information-Theoretic Lower Bounds , 2010, IEEE Transactions on Information Theory.

[20]  Kamiar Rahnama Rad Nearly Sharp Sufficient Conditions on Exact Sparsity Pattern Recovery , 2009, IEEE Transactions on Information Theory.

[21]  Georgios B. Giannakis,et al.  Compressed Sensing for Wideband Cognitive Radios , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[22]  D. Donoho For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[23]  Vahid Tarokh,et al.  Shannon-Theoretic Limits on Noisy Compressive Sampling , 2007, IEEE Transactions on Information Theory.

[24]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[25]  Yue Wang,et al.  Performance analysis of partial support recovery and signal reconstruction of compressed sensing , 2014, IET Signal Process..

[26]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[27]  Rui M. Castro,et al.  Adaptive Compressed Sensing for Support Recovery of Structured Sparse Sets , 2014, IEEE Transactions on Information Theory.

[28]  Mikael Skoglund,et al.  Projection-Based and Look-Ahead Strategies for Atom Selection , 2011, IEEE Transactions on Signal Processing.

[29]  Mikael Skoglund,et al.  A Measurement Rate-MSE Tradeoff for Compressive Sensing Through Partial Support Recovery , 2014, IEEE Transactions on Signal Processing.

[30]  Jarvis D. Haupt,et al.  Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[31]  Gongguo Tang,et al.  Performance Analysis for Sparse Support Recovery , 2009, IEEE Transactions on Information Theory.

[32]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[33]  Richard G. Baraniuk,et al.  An Architecture for Compressive Imaging , 2006, 2006 International Conference on Image Processing.

[34]  Sungho Tak,et al.  A Data-Driven Sparse GLM for fMRI Analysis Using Sparse Dictionary Learning With MDL Criterion , 2011, IEEE Transactions on Medical Imaging.

[35]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[36]  Vahid Tarokh,et al.  Noisy compressive sampling limits in linear and sublinear regimes , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[37]  Sundeep Rangan,et al.  Necessary and Sufficient Conditions for Sparsity Pattern Recovery , 2008, IEEE Transactions on Information Theory.

[38]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[39]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[40]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[41]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[42]  P. P. Vaidyanathan,et al.  Nested Arrays: A Novel Approach to Array Processing With Enhanced Degrees of Freedom , 2010, IEEE Transactions on Signal Processing.

[43]  J. Haupt,et al.  Compressive wireless sensing , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[44]  Sergio Verdú,et al.  Optimal Phase Transitions in Compressed Sensing , 2011, IEEE Transactions on Information Theory.

[45]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[46]  Robert D. Nowak,et al.  Compressed Channel Sensing: A New Approach to Estimating Sparse Multipath Channels , 2010, Proceedings of the IEEE.

[47]  Dave Zachariah,et al.  Dynamic Iterative Pursuit , 2012, IEEE Transactions on Signal Processing.

[48]  David P. Wipf,et al.  A unified Bayesian framework for MEG/EEG source imaging , 2009, NeuroImage.