Compressed Sensing of Simultaneous Low-Rank and Joint-Sparse Matrices

In this paper we consider the problem of recovering a high dimensional data matrix from a set of incomplete and noisy linear measurements. We introduce a new model that can efficiently restrict the degrees of freedom of the problem and is generic enough to find a lot of applications, for instance in multichannel signal compressed sensing (e.g. sensor networks, hyperspectral imaging) and compressive sparse principal component analysis (s-PCA). We assume data matrices have a simultaneous low-rank and joint sparse structure, and we propose a novel approach for efficient compressed sensing (CS) of such data. Our CS recovery approach is based on a convex minimization problem that incorporates this restrictive structure by jointly regularizing the solutions with their nuclear (trace) norm and l2/l1 mixed norm. Our theoretical analysis uses a new notion of restricted isometry property (RIP) and shows that, for sampling schemes satisfying RIP, our approach can stably recover all low-rank and joint-sparse matrices. For a certain class of random sampling schemes satisfying a particular concentration bound (e.g. the subgaussian ensembles) we derive a lower bound on the number of CS measurements indicating the near-optimality of our recovery approach as well as a significant enhancement compared to the state-of-the-art. We introduce an iterative algorithm based on proximal calculus in order to solve the joint nuclear and l2/l1 norms minimization problem and, finally, we illustrate the empirical recovery phase transition of this approach by series of numerical experiments.

[1]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[2]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[3]  John Wright,et al.  Principal Component Pursuit with reduced linear measurements , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[4]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[5]  Alexandre d'Aspremont,et al.  Optimal Solutions for Sparse Principal Component Analysis , 2007, J. Mach. Learn. Res..

[6]  Pierre Vandergheynst,et al.  Joint trace/TV norm minimization: A new efficient approach for spectral compressive imaging , 2012, 2012 19th IEEE International Conference on Image Processing.

[7]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[8]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, NIPS 2004.

[9]  Yonina C. Eldar,et al.  Rank Awareness in Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[10]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[11]  Pierre Vandergheynst,et al.  Distributed compressed sensing for sensor networks using thresholding , 2009, Optical Engineering + Applications.

[12]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[13]  Jong Chul Ye,et al.  Compressive MUSIC: A Missing Link Between Compressive Sensing and Array Signal Processing , 2010, ArXiv.

[14]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[15]  Emmanuel J. Candès,et al.  Tight oracle bounds for low-rank matrix recovery from a minimal number of random measurements , 2010, ArXiv.

[16]  E.J. Candes Compressive Sampling , 2022 .

[17]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[18]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[19]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[20]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[21]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[22]  Mohamed-Jalal Fadili,et al.  Monotone operator splitting for optimization problems in sparse recovery , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[23]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[24]  Pierre Vandergheynst,et al.  Hyperspectral image compressed sensing via low-rank and joint-sparse matrix recovery , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[26]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[27]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[28]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[29]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[30]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[31]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[32]  Pierre Vandergheynst,et al.  Guaranteed recovery of a low-rank and joint-sparse matrix from incomplete and noisy measurements (Abstract) , 2011 .

[33]  Stephen P. Boyd,et al.  Rank minimization and applications in system theory , 2004, Proceedings of the 2004 American Control Conference.

[34]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.