Support Recovery of Sparse Signals in the Presence of Multiple Measurement Vectors

This paper studies the performance limits in the support recovery of sparse signals based on multiple measurement vectors (MMV). An information-theoretic analytical framework inspired by the connection to the single-input multiple-output multiple-access channel communication is established to reveal the performance limits in the support recovery of sparse signals with fixed number of nonzero entries. Sharp sufficient and necessary conditions for asymptotically successful support recovery are derived in terms of the number of measurements per vector, the number of nonzero rows, the measurement noise level, and the number of measurement vectors. Through the interpretations of the results, the benefit of having MMV for sparse signal recovery is illustrated, thus providing a theoretical foundation to the performance improvement enabled by MMV as observed in many existing simulation results. In particular, it is shown that the structure (rank) of the matrix formed by the nonzero entries plays an important role in the performance limits of support recovery.

[1]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[2]  E.J. Candes.  Compressive Sampling , 2022 .

[3]  Galen Reeves,et al.  Sampling bounds for sparse support recovery in the presence of noise , 2008, 2008 IEEE International Symposium on Information Theory.

[4]  Thomas W. Parks,et al.  Extrapolation and spectral estimation with iterative weighted norm modification , 1991, IEEE Trans. Signal Process..

[5]  Joel A. Tropp,et al.  Simultaneous sparse approximation via greedy pursuit , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  David P. Wipf,et al.  A unified Bayesian framework for MEG/EEG source imaging , 2009, NeuroImage.

[7]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[8]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[9]  Bhaskar D. Rao,et al.  Insights into the stable recovery of sparse solutions in overcomplete representations using network information theory , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices , 2008, IEEE Transactions on Information Theory.

[11]  Robert D. Nowak,et al.  Compressed channel sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[12]  D. Bernstein.  Matrix Mathematics: Theory, Facts, and Formulas , 2009 .

[13]  David Tse,et al.  Fundamentals of Wireless Communication , 2005 .

[14]  Amos Lapidoth,et al.  Nearest neighbor decoding for additive non-Gaussian noise channels , 1996, IEEE Trans. Inf. Theory.

[15]  A. Ioannides,et al.  Continuous probabilistic solutions to the biomagnetic inverse problem , 1990 .

[16]  Richard G. Baraniuk,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[17]  Yoshikatsu Watanabe.  On the ω2 distribution , 1952 .

[18]  Rohit U. Nabar,et al.  Introduction to Space-Time Wireless Communications , 2003 .

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[21]  Yoram Bresler,et al.  A new algorithm for computing sparse solutions to linear inverse problems , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[22]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity , 2008, ArXiv.

[23]  Donald L. Duttweiler,et al.  Proportionate normalized least-mean-squares adaptation in echo cancelers , 2000, IEEE Trans. Speech Audio Process..

[24]  Yoram Bresler,et al.  Globally convergent edge-preserving regularized reconstruction: an application to limited-angle tomography , 1998, IEEE Trans. Image Process..

[25]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Babak Hassibi,et al.  How much training is needed in multiple-antenna wireless links? , 2003, IEEE Trans. Inf. Theory.

[27]  Vahid Tarokh,et al.  Shannon-Theoretic Limits on Noisy Compressive Sampling , 2007, IEEE Transactions on Information Theory.

[28]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[29]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[30]  I F Gorodnitsky,et al.  Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm. , 1995, Electroencephalography and clinical neurophysiology.

[31]  Martin J. Wainwright,et al.  Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting , 2009, IEEE Trans. Inf. Theory.

[32]  Bhaskar D. Rao,et al.  Support Recovery of Sparse Signals , 2010, ArXiv.

[33]  J. W. Silverstein.  The Smallest Eigenvalue of a Large Dimensional Wishart Matrix , 1985 .

[34]  Sundeep Rangan,et al.  Necessary and Sufficient Conditions for Sparsity Pattern Recovery , 2008, IEEE Transactions on Information Theory.

[35]  Jonathan R. Partington,et al.  Linear algebra in action (Graduate Studies in Mathematics 78) , 2008 .

[36]  D. Bernstein.  Matrix Mathematics: Theory, Facts, and Formulas , 2009 .

[37]  Ping Feng,et al.  Spectrum-blind minimum-rate sampling and reconstruction of multiband signals , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[38]  Andrzej Cichocki,et al.  Blind source separation: new tools for extraction of source signals and denoising , 2005, SPIE Defense + Commercial Sensing.

[39]  Gongguo Tang,et al.  Performance Analysis for Sparse Support Recovery , 2009, IEEE Transactions on Information Theory.

[40]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[41]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[42]  S. Trione.  On The Distribution , 2002 .

[43]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[44]  Bhaskar D. Rao,et al.  Sparse channel estimation via matching pursuit with application to equalization , 2002, IEEE Trans. Commun..

[45]  Y. Bresler,et al.  Spectrum-blind minimum-rate sampling and reconstruction of 2-D multiband signals , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[46]  Yoram Bresler,et al.  Subspace Methods for Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[47]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[48]  Enrico Magli,et al.  Distributed Compressed Sensing , 2015 .

[49]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[50]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[51]  Bhaskar D. Rao,et al.  Sparse signal recovery in the presence of correlated multiple measurement vectors , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[52]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[53]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[54]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[55]  Yonina C. Eldar,et al.  GPS signal acquisition via compressive multichannel sampling , 2011, Phys. Commun..

[56]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[57]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[58]  Michael I. Jordan,et al.  Union support recovery in high-dimensional multivariate regression , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[59]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[60]  Yoram Bresler,et al.  Subspace-augmented MUSIC for joint sparse recovery with any rank , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[61]  Ping Feng,et al.  Universal Minimum-Rate Sampling and Spectrum-Blind Reconstruction for Multiband Signals , 1998 .

[62]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[63]  Yoram Bresler,et al.  On the necessary density for spectrum-blind nonuniform sampling subject to quantization , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[64]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[65]  Andrzej Cichocki,et al.  Improved M-FOCUSS Algorithm With Overlapping Blocks for Locally Smooth Sparse Signals , 2008, IEEE Transactions on Signal Processing.

[66]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[68]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[69]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[70]  Andrea J. Goldsmith,et al.  Capacity limits of MIMO channels , 2003, IEEE J. Sel. Areas Commun..

[71]  Bhaskar D. Rao,et al.  Algorithms for robust linear regression by exploiting the connection to sparse signal recovery , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[72]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[73]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[74]  Bhaskar D. Rao,et al.  Limits on Support Recovery of Sparse Signals via Multiple-Access Communication Techniques , 2011, IEEE Transactions on Information Theory.

[75]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[76]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[77]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[78]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[79]  Brian D. Jeffs.  Sparse inverse solution methods for signal and image processing applications , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[80]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[81]  Esau Ruiz Gaistardo.  Neighbor discovery in ad-hoc networks , 2015 .

[82]  Bhaskar D. Rao,et al.  Adaptive filtering algorithms for promoting sparsity , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[83]  Peter Bühlmann.  Regression shrinkage and selection via the Lasso: a retrospective (Robert Tibshirani): Comments on the presentation , 2011 .

[84]  Harinath Garudadri,et al.  Low complexity sensors for body area networks , 2010, PETRA '10.

[85]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[86]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[87]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[88]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[89]  Wai C. Chu,et al.  Speech Coding Algorithms , 2003 .

[90]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.