The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing

This paper deals with the computational complexity of conditions which guarantee that the NP-hard problem of finding the sparsest solution to an underdetermined linear system can be solved by efficient algorithms. In the literature, several such conditions have been introduced. The most well-known ones are the mutual coherence, the restricted isometry property (RIP), and the nullspace property (NSP). While evaluating the mutual coherence of a given matrix is easy, it has been suspected for some time that evaluating RIP and NSP is computationally intractable in general. We confirm these conjectures by showing that for a given matrix A and positive integer k, computing the best constants for which the RIP or NSP hold is, in general, NP-hard. These results are based on the fact that determining the spark of a matrix is NP-hard, which is also established in this paper. Furthermore, we also give several complexity statements about problems related to the above concepts.

[1]  Marc Teboulle,et al.  Conditional Gradient Algorithmsfor Rank-One Matrix Approximations with a Sparsity Constraint , 2011, SIAM Rev..

[2]  M. A. Iwen.  Simple deterministically constructible RIP matrices with sublinear fourier sampling requirements , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[3]  João Filipe Queiró,et al.  On the interlacing property for singular values and eigenvalues , 1987 .

[4]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[5]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[6]  S. T. McCormick,et al.  A combinatorial approach to some sparse matrix problems , 1983 .

[7]  S. De Marchi,et al.  Generalized Vandermonde determinants, Toeplitz matrices and Schur functions , 1999 .

[8]  T. Coleman,et al.  The null space problem I. complexity , 1986 .

[9]  Pascal Koiran,et al.  Hidden Cliques and the Certification of the Restricted Isometry Property , 2012, IEEE Transactions on Information Theory.

[10]  Willem H. Haemers,et al.  Spectra of Graphs , 2011 .

[11]  Holly P. Hirst,et al.  Bounding the Roots of Polynomials , 1997 .

[12]  C. V. Nuffelen.  On the incidence matrix of a graph , 1976 .

[13]  Lie Wang,et al.  New Bounds for Restricted Isometry Constants , 2009, IEEE Transactions on Information Theory.

[14]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[15]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[16]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[17]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[18]  Weiyu Xu,et al.  Precisely Verifying the Null Space Conditions in Compressed Sensing: A Sandwiching Algorithm , 2013, ArXiv.

[19]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[20]  Pascal Koiran,et al.  On the Certification of the Restricted Isometry Property , 2011, ArXiv.

[21]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[22]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[23]  Hans Peter Schlickewei,et al.  Generalized Vandermonde determinants , 2000 .

[24]  P. Rigollet,et al.  Optimal detection of sparse principal components in high dimension , 2012, 1202.5070.

[25]  Jeffrey D. Blanchard,et al.  COMPRESSED SENSING : HOW SHARP IS THE RIP ? , 2009 .

[26]  Dustin G. Mixon,et al.  Certifying the Restricted Isometry Property is Hard , 2012, IEEE Transactions on Information Theory.

[27]  Jens Vygen,et al.  The Book Review Column1 , 2020, SIGACT News.

[28]  Alexander Vardy,et al.  The intractability of computing the minimum distance of a code , 1997, IEEE Trans. Inf. Theory.

[29]  Alon Itai,et al.  Finding a Minimum Circuit in a Graph , 1978, SIAM J. Comput..

[30]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.

[31]  E. Heineman.  Generalized Vandermonde determinants , 1929 .

[32]  Bernhard Korte,et al.  Combinatorial Optimization , 1992, NATO ASI Series.

[33]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[34]  Stephen J. Dilworth,et al.  Explicit constructions of RIP matrices and related problems , 2010, ArXiv.

[35]  Shmuel Friedland.  Bounds on the spectral radius of graphs with e edges , 1988 .

[36]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[37]  Richard M. Karp,et al.  Reducibility among combinatorial problems" in complexity of computer computations , 1972 .

[38]  Arkadi Nemirovski,et al.  On verifiable sufficient conditions for sparse signal recovery via ℓ1 minimization , 2008, Math. Program..

[39]  Dominic Welsh,et al.  On the Vector Representation of Matroids , 1970 .

[40]  Rémi Gribonval,et al.  Restricted Isometry Constants Where $\ell ^{p}$ Sparse Recovery Can Fail for $0≪ p \leq 1$ , 2009, IEEE Transactions on Information Theory.

[41]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[42]  Leonid Khachiyan,et al.  On the Complexity of Approximating Extremal Determinants in Matrices , 1995, J. Complex..

[43]  Jared Tanner,et al.  Explorer Compressed Sensing : How Sharp Is the Restricted Isometry Property ? , 2011 .

[44]  Alexandre d'Aspremont,et al.  Testing the nullspace property using semidefinite programming , 2008, Math. Program..

[45]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[46]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[47]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[48]  Philippe Rigollet,et al.  Computational Lower Bounds for Sparse PCA , 2013, ArXiv.

[49]  Leonid Gurvits,et al.  Vandermonde Matrices, NP-Completeness, and Transversal Subspaces , 2003, Found. Comput. Math..

[50]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[51]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[52]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[53]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, SIAM Rev..

[54]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[55]  Dustin G. Mixon,et al.  Full Spark Frames , 2011, 1110.3548.

[56]  J. G. Pierce,et al.  Geometric Algorithms and Combinatorial Optimization , 2016 .

[57]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[58]  Edoardo Amaldi,et al.  The Complexity and Approximability of Finding Maximum Feasible Subsystems of Linear Relations , 1995, Theor. Comput. Sci..

[59]  Otmar Scherzer,et al.  Handbook of Mathematical Methods in Imaging , 2015, Handbook of Mathematical Methods in Imaging.

[60]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[61]  Alon Itai,et al.  Finding a minimum circuit in a graph , 1977, STOC '77.

[62]  Jared Tanner,et al.  Improved Bounds on Restricted Isometry Constants for Gaussian Matrices , 2010, SIAM J. Matrix Anal. Appl..

[63]  Davies Rémi Gribonval.  Restricted Isometry Constants Where Lp Sparse Recovery Can Fail for 0 , 2008 .

[64]  Alexandre d'Aspremont,et al.  Optimal Solutions for Sparse Principal Component Analysis , 2007, J. Mach. Learn. Res..

[65]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[66]  Philip E. Gill,et al.  Numerical Linear Algebra and Optimization , 1991 .

[67]  Yoram Bresler,et al.  Computing performance guarantees for compressed sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[68]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..