The Restricted Isometry Property for Random Block Diagonal Matrices

In Compressive Sensing, the Restricted Isometry Property (RIP) ensures that robust recovery of sparse vectors is possible from noisy, undersampled measurements via computationally tractable algorithms. It is by now well-known that Gaussian (or, more generally, sub-Gaussian) random matrices satisfy the RIP under certain conditions on the number of measurements. Their use can be limited in practice, however, due to storage limitations, computational considerations, or the mismatch of such matrices with certain measurement architectures. These issues have recently motivated considerable effort towards studying the RIP for structured random matrices. In this paper, we study the RIP for block diagonal measurement matrices where each block on the main diagonal is itself a sub-Gaussian random matrix. Our main result states that such matrices can indeed satisfy the RIP but that the requisite number of measurements depends on certain properties of the basis in which the signals are sparse. In the best case, these matrices perform nearly as well as dense Gaussian random matrices, despite having many fewer nonzero entries.

[1]  Holger Rauhut,et al.  Compressive Sensing with structured random matrices , 2012 .

[2]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[3]  M. Talagrand.  The Generic Chaining , 2005 .

[4]  Rachel Ward,et al.  New and Improved Johnson-Lindenstrauss Embeddings via the Restricted Isometry Property , 2010, SIAM J. Math. Anal..

[5]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[6]  Justin K. Romberg,et al.  Restricted Isometries for Partial Random Circulant Matrices , 2010, ArXiv.

[7]  Richard G. Baraniuk,et al.  Signal Processing With Compressive Measurements , 2010, IEEE Journal of Selected Topics in Signal Processing.

[8]  Jared Tanner,et al.  Improved Bounds on Restricted Isometry Constants for Gaussian Matrices , 2010, SIAM J. Matrix Anal. Appl..

[9]  Henry Arguello,et al.  Restricted Isometry Property in coded aperture compressive spectral imaging , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[10]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.

[11]  Michael B. Wakin,et al.  The Restricted Isometry Property for block diagonal matrices , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[12]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[13]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[14]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[15]  Tyrone L. Vincent,et al.  On the observability of linear systems from random, compressive measurements , 2010, 49th IEEE Conference on Decision and Control (CDC).

[16]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[17]  E.J. Candes.  Compressive Sampling , 2022 .

[18]  Christopher J. Rozell,et al.  Dynamic filtering of sparse signals using reweighted ℓ1 , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  David L. Donoho,et al.  Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[20]  Marc E. Pfetsch,et al.  The Computational Complexity of RIP, NSP, and Related Concepts in Compressed Sensing , 2012, ArXiv.

[21]  Jared Tanner,et al.  Phase Transitions for Greedy Sparse Approximation Algorithms , 2010, ArXiv.

[22]  Jubo Zhu,et al.  Recovery of sparse signals using OMP and its variants: convergence analysis based on RIP , 2011 .

[23]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[24]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[25]  Robert D. Nowak,et al.  Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation , 2010, IEEE Transactions on Information Theory.

[26]  Justin K. Romberg,et al.  Sparsity penalties in dynamical system estimation , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[27]  Sina Jafarpour,et al.  Deterministic matrices matching the compressed sensing phase transitions of Gaussian random matrices , 2012, Proceedings of the National Academy of Sciences.

[28]  Massimo Fornasier,et al.  Compressive Sensing and Structured Random Matrices , 2010 .

[29]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[30]  Trac D. Tran,et al.  Fast and Efficient Compressive Sensing Using Structurally Random Matrices , 2011, IEEE Transactions on Signal Processing.

[31]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[32]  Jared Tanner,et al.  Explorer Compressed Sensing : How Sharp Is the Restricted Isometry Property ? , 2011 .

[33]  Michael B. Wakin,et al.  Concentration of measure for block diagonal matrices with repeated blocks , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[34]  Justin K. Romberg,et al.  Estimation and dynamic updating of time-varying signals with sparse variations , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Michael B. Wakin,et al.  Concentration of measure for block diagonal measurement matrices , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Michael B. Wakin,et al.  Stable manifold embeddings with operators satisfying the Restricted Isometry Property , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[38]  Holger Rauhut,et al.  Suprema of Chaos Processes and the Restricted Isometry Property , 2012, ArXiv.

[39]  Michael B. Wakin,et al.  Concentration of Measure for Block Diagonal Matrices With Applications to Compressive Signal Processing , 2011, IEEE Transactions on Signal Processing.

[40]  Christopher J. Rozell,et al.  Stable Takens' Embeddings for Linear Dynamical Systems , 2010, IEEE Transactions on Signal Processing.

[41]  Yurii Nesterov,et al.  Generalized Power Method for Sparse Principal Component Analysis , 2008, J. Mach. Learn. Res..

[42]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[43]  H. Rauhut.  Compressive Sensing and Structured Random Matrices , 2009 .

[44]  Marc E. Pfetsch,et al.  The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing , 2012, IEEE Transactions on Information Theory.

[45]  G. Peyré,et al.  A numerical exploration of compressed sampling recovery , 2010 .

[46]  S. Foucart.  Sparse Recovery Algorithms: Sufficient Conditions in Terms of RestrictedIsometry Constants , 2012 .

[47]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[48]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .