Rank Awareness in Joint Sparse Recovery

This paper revisits the sparse multiple measurement vector (MMV) problem, where the aim is to recover a set of jointly sparse multichannel vectors from incomplete measurements. This problem is an extension of single channel sparse recovery, which lies at the heart of compressed sensing. Inspired by the links to array signal processing, a new family of MMV algorithms is considered that highlight the role of rank in determining the difficulty of the MMV recovery problem. The simplest such method is a discrete version of MUSIC which is guaranteed to recover the sparse vectors in the full rank MMV setting, under mild conditions. This idea is extended to a rank aware pursuit algorithm that naturally reduces to Order Recursive Matching Pursuit (ORMP) in the single measurement case while also providing guaranteed recovery in the full rank setting. In contrast, popular MMV methods such as Simultaneous Orthogonal Matching Pursuit (SOMP) and mixed norm minimization techniques are shown to be rank blind in terms of worst case analysis. Numerical simulations demonstrate that the rank aware techniques are significantly better than existing methods in dealing with multiple measurements.

[1]  J. Tropp.  Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[2]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[3]  Minh N. Do,et al.  A Theory for Sampling Signals from a Union of Subspaces , 2022 .

[4]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[5]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[6]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[7]  Pierre Vandergheynst,et al.  Average Case Analysis of Multichannel Thresholding , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[8]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[9]  G. Stewart.  Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems , 1973 .

[10]  H. Rauhut,et al.  Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms , 2008 .

[11]  Ping Feng,et al.  Spectrum-blind minimum-rate sampling and reconstruction of multiband signals , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[12]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[13]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[14]  Yonina C. Eldar.  Compressed Sensing of Analog Signals in Shift-Invariant Spaces , 2008, IEEE Transactions on Signal Processing.

[15]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[16]  Yoram Bresler,et al.  iMUSIC: Iterative MUSIC Algorithm for Joint Sparse Recovery with Any Rank , 2010, ArXiv.

[17]  P. Stoica,et al.  Improved sequential MUSIC , 1995, IEEE Transactions on Aerospace and Electronic Systems.

[18]  Richard M. Leahy,et al.  Source localization using recursively applied and projected (RAP) MUSIC , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[19]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[20]  Yonina C. Eldar,et al.  From Theory to Practice: Sub-Nyquist Sampling of Sparse Wideband Analog Signals , 2009, IEEE Journal of Selected Topics in Signal Processing.

[21]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[22]  Yonina C. Eldar,et al.  Blind Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[23]  Yonina C. Eldar,et al.  Low Rate Sampling Schemes for Time Delay Estimation , 2009, ArXiv.

[24]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[25]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[26]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[27]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[28]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[29]  Albert Fannjiang,et al.  The MUSIC algorithm for sparse objects: a compressed sensing analysis , 2010, ArXiv.

[30]  Richard G. Baraniuk,et al.  Distributed Compressive Sensing , 2009, ArXiv.

[31]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[32]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[33]  Don H. Johnson,et al.  Array Signal Processing: Concepts and Techniques , 1993 .

[34]  V. Rich.  Personal communication , 1989, Nature.

[35]  Jong Chul Ye,et al.  Compressive MUSIC: A Missing Link Between Compressive Sensing and Array Signal Processing , 2010, ArXiv.

[36]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[37]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[38]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[39]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[40]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[41]  Michael P. Friedlander,et al.  Theoretical and Empirical Results for Recovery From Multiple Measurements , 2009, IEEE Transactions on Information Theory.

[42]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[43]  Harry L. Van Trees,et al.  Optimum Array Processing , 2002 .

[44]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[45]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[46]  Chong Kwan Un,et al.  A Sequential Estimation Approach for Performance Improvement of Eigenstructure-Based Methods in Array Processing , 1993, IEEE Trans. Signal Process..

[47]  T. Blumensath,et al.  On the Difference Between Orthogonal Matching Pursuit and Orthogonal Least Squares , 2007 .

[48]  Yonina C. Eldar,et al.  Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals , 2007, IEEE Transactions on Signal Processing.

[49]  Jong Chul Ye,et al.  Compressive MUSIC: Revisiting the Link Between Compressive Sensing and Array Signal Processing , 2012, IEEE Trans. Inf. Theory.

[50]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[51]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[52]  Vladimir N. Temlyakov,et al.  Vector greedy algorithms , 2003, J. Complex..

[53]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[54]  Ping Feng,et al.  Universal Minimum-Rate Sampling and Spectrum-Blind Reconstruction for Multiband Signals , 1998 .

[55]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[56]  Yonina C. Eldar,et al.  Xampling: Analog to digital at sub-Nyquist rates , 2009, IET Circuits Devices Syst..