Theoretical Results on Sparse Representations of Multiple-Measurement Vectors

The sparse representation of a multiple-measurement vector (MMV) is a relatively new problem in sparse representation. Efficient methods have been proposed. Although many theoretical results that are available in a simple case-single-measurement vector (SMV)-the theoretical analysis regarding MMV is lacking. In this paper, some known results of SMV are generalized to MMV. Some of these new results take advantages of additional information in the formulation of MMV. We consider the uniqueness under both an lscr0-norm-like criterion and an lscr1-norm-like criterion. The consequent equivalence between the lscr0-norm approach and the lscr1-norm approach indicates a computationally efficient way of finding the sparsest representation in a redundant dictionary. For greedy algorithms, it is proven that under certain conditions, orthogonal matching pursuit (OMP) can find the sparsest representation of an MMV with computational efficiency, just like in SMV. Simulations show that the predictions made by the proved theorems tend to be very conservative; this is consistent with some recent advances in probabilistic analysis based on random matrix theory. The connections will be discussed

[1]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[2]  I F Gorodnitsky,et al.  Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm. , 1995, Electroencephalography and clinical neurophysiology.

[3]  Tiefeng Jiang,et al.  The asymptotic distributions of the largest entries of sample correlation matrices , 2004, math/0406184.

[4]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[5]  Yoram Bresler,et al.  On the Optimality of the Backward Greedy Algorithm for the Subset Selection Problem , 2000, SIAM J. Matrix Anal. Appl..

[6]  V. Temlyakov.  A remark on simultaneous greedy approximation , 2004 .

[7]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[8]  Joel A. Tropp,et al.  Simultaneous sparse approximation via greedy pursuit , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[9]  Dany Leviatan,et al.  Simultaneous approximation by greedy algorithms , 2006, Adv. Comput. Math..

[10]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[11]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[12]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..

[13]  X. Huo,et al.  Theoretical Results About Finding the Sparsest Representations of Multiple Measurement Vectors ( MMV ) in an Over-complete Dictionary , Using ` 1-Norm Minimization and Greedy Algorithms , 2004 .

[14]  Yoram Bresler,et al.  Fast optimal and suboptimal algorithms for sparse solutions to linear inverse problems , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[15]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[16]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[17]  Vladimir N. Temlyakov,et al.  Vector greedy algorithms , 2003, J. Complex..

[18]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[19]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[20]  Gene H. Golub,et al.  Matrix computations , 1983 .

[21]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[22]  Jie Chen,et al.  Sparse representations for multiple measurement vectors (MMV) in an over-complete dictionary , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[23]  Tiefeng Jiang,et al.  Maxima of entries of Haar distributed matrices , 2005 .

[24]  J. Tropp.  Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[25]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[26]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[27]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[28]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[29]  Bhaskar D. Rao,et al.  Diversity measure minimization based method for computing sparse solutions to linear inverse problems with multiple measurement vectors , 2004, ICASSP.

[30]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[31]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[32]  Dany Leviatan,et al.  Simultaneous greedy approximation in Banach spaces , 2005, J. Complex..

[33]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[34]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[35]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Yoram Bresler,et al.  A new algorithm for computing sparse solutions to linear inverse problems , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[37]  J. Tropp.  JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION , 2004 .

[38]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[39]  Bhaskar D. Rao,et al.  Measures and algorithms for best basis selection , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[40]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[41]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[42]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[43]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.