Testing Positive Semi-Definiteness via Random Submatrices

We study the problem of testing whether a matrix $\mathrm{A}\in \mathbb{R}^{n\times n}$ with bounded entries ($\Vert \mathrm{A}\Vert_{\infty}\leq 1$) is positive semidefinite (PSD), or $\epsilon$-far in Euclidean distance from the PSD cone, meaning that $\min\nolimits_{\mathrm{B}\succeq 0}\Vert \mathrm{A}-\mathrm{B}\Vert_{F}^{2} > \epsilon n^{2}$, where $\mathrm{B}\succeq 0$ denotes that B is PSD. Our main algorithmic contribution is a non-adaptive tester which distinguishes between these cases using only $\tilde{O}(1/\epsilon^{4})$ queries to the entries of A.11Throughout the paper, $\tilde{O}(\cdot)$ hides $\log(1/\epsilon)$ factors. If instead of the Eucledian norm we considered the distance in spectral norm, we obtain the “$\ell_{\infty}$-gap problem”, where A is either PSD or satisfies $\min\nolimits_{\mathrm{B}\succ 0}\Vert \mathrm{A}-\mathrm{B}\Vert_{2} > \epsilon n$. For this related problem, we give a $\tilde{O}(1/\epsilon^{2})$ query tester, which we show is optimal up to $\log(1/\epsilon)$ factors. Both our testers randomly sample a collection of principal sub-matrices and check whether these sub-matrices are PSD. Consequentially, our algorithms achieve one-sided error: whenever they output that A is not PSD, they return a certificate that A has negative eigenvalues. We complement our upper bound for PSD testing with Eucledian norm distance by giving a $\tilde{\Omega}(1/\epsilon^{2})$ lower bound for any non-adaptive algorithm. Our lower bound construction is general, and can be used to derive lower bounds for a number of spectral testing problems. As an example of the applicability of our construction, we obtain a new $\tilde{\Omega}(1/\epsilon^{4})$ sampling lower bound for testing the Schatten-1 norm with a $\epsilon n^{1.5}$ gap, extending a result of Balcan, Li, Woodruff, and Zhang [11]. In addition, our hard instance results in new sampling lower bounds for estimating the Ky-Fan Norm, and the cost of rank-$k$ approximations, i.e. $\Vert\mathrm{A}-\mathrm{A}_{k}\Vert_{F}^{2}=\sum\nolimits_{i > k}\sigma_{i}^{2}(\mathrm{A})$.

[1]  Hossein Jowhari,et al.  Tight bounds for Lp samplers, finding duplicates in streams, and related problems , 2010, PODS.

[2]  Ori Sasson,et al.  Property testing of data dimensionality , 2003, SODA '03.

[3]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[4]  Dana Ron,et al.  Testing metric properties , 2003, Inf. Comput..

[5]  David P. Woodruff,et al.  On approximating functions of the singular values in a stream , 2016, STOC.

[6]  David P. Woodruff,et al.  Low rank approximation and regression in input sparsity time , 2012, STOC '13.

[7]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[8]  Alex Gittens,et al.  TAIL BOUNDS FOR ALL EIGENVALUES OF A SUM OF RANDOM MATRICES , 2011, 1104.4513.

[9]  Arnab Bhattacharyya,et al.  Testing Sparsity over Known and Unknown Bases , 2016, ICML.

[10]  J. Tropp Norms of Random Submatrices and Sparse Approximation , 2008 .

[11]  David P. Woodruff,et al.  On Sketching Matrix Norms and the Top Singular Vector , 2014, SODA.

[12]  David P. Woodruff,et al.  1-pass relative-error Lp-sampling with applications , 2010, SODA '10.

[13]  David P. Woodruff,et al.  Improved testing of low rank matrices , 2014, KDD.

[14]  Kyle Luh,et al.  Four Deviations Suffice for Rank 1 Matrices , 2019, ArXiv.

[15]  David P. Woodruff,et al.  Querying a Matrix through Matrix-Vector Products , 2019, ICALP.

[16]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[17]  Robert Krauthgamer,et al.  Schatten Norms in Matrix Streams: Hello Sparsity, Goodbye Dimension , 2019, ICML.

[18]  David P. Woodruff,et al.  On the exact space complexity of sketching and streaming small norms , 2010, SODA '10.

[19]  Satish Rao,et al.  Expander flows, geometric embeddings and graph partitioning , 2004, STOC '04.

[20]  David P. Woodruff,et al.  Tight Bounds for Sketching the Operator Norm, Schatten Norms, and Subspace Embeddings , 2022, APPROX-RANDOM.

[21]  Sanjeev Arora,et al.  Polynomial time approximation schemes for Euclidean traveling salesman and other geometric problems , 1998, JACM.

[22]  Shang-Hua Teng,et al.  Spectral sparsification of graphs: theory and algorithms , 2013, CACM.

[23]  A. Turing ROUNDING-OFF ERRORS IN MATRIX PROCESSES , 1948 .

[24]  T. Tao Topics in Random Matrix Theory , 2012 .

[25]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[26]  David P. Woodruff,et al.  On Approximating Matrix Norms in Data Streams , 2019, SIAM J. Comput..

[27]  David P. Woodruff,et al.  Embeddings of Schatten Norms with Applications to Data Streams , 2017, ICALP.

[28]  David P. Woodruff,et al.  Sublinear Time Low-Rank Approximation of Positive Semidefinite Matrices , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[29]  Fan R. K. Chung,et al.  Lectures on Spectral Graph Theory , 2001 .

[30]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[31]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[32]  R. Saigal,et al.  Handbook of semidefinite programming : theory, algorithms, and applications , 2000 .

[33]  David P. Woodruff,et al.  Optimal Sketching for Kronecker Product Regression and Low Rank Approximation , 2019, NeurIPS.

[34]  T. Blumensath,et al.  Theory and Applications , 2011 .

[35]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[36]  James Demmel,et al.  Fast matrix multiplication is stable , 2006, Numerische Mathematik.

[37]  Oded Goldreich,et al.  Introduction to Property Testing , 2017 .

[38]  David P. Woodruff,et al.  Testing Matrix Rank, Optimally , 2018, SODA.

[39]  David P. Woodruff,et al.  Robust and Sample Optimal Algorithms for PSD Low Rank Approximation , 2020, 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).

[40]  David P. Woodruff,et al.  Sublinear Time Low-Rank Approximation of Distance Matrices , 2018, NeurIPS.

[41]  Dana Ron,et al.  Property testing and its connection to learning and approximation , 1998, JACM.

[42]  Martin J. Wainwright,et al.  High-Dimensional Statistics , 2019 .

[43]  Noga Alon,et al.  The space complexity of approximating the frequency moments , 1996, STOC '96.

[44]  David P. Woodruff,et al.  Perfect Lp Sampling in a Data Stream , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[45]  David P. Woodruff,et al.  Sample-Optimal Low-Rank Approximation of Distance Matrices , 2019, COLT.

[46]  Yin Tat Lee,et al.  A matrix expander Chernoff bound , 2017, STOC.

[47]  Oded Goldreich,et al.  Introduction to Testing Graph Properties , 2010, Property Testing.

[48]  G. Pisier,et al.  THE NON-COMMUTATIVE KHINTCHINE INEQUALITIES FOR $0 , 2014, Journal of the Institute of Mathematics of Jussieu.

[49]  David Steurer,et al.  Fast SDP algorithms for constraint satisfaction problems , 2010, SODA '10.

[50]  David P. Woodruff,et al.  Weighted Reservoir Sampling from Distributed Streams , 2019, PODS.

[51]  Florent Krzakala,et al.  Spectral Clustering of graphs with the Bethe Hessian , 2014, NIPS.

[52]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[53]  R. C. Thompson Principal submatrices IX: Interlacing inequalities for singular values of submatrices , 1972 .

[54]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[55]  Alexandr Andoni,et al.  Eigenvalues of a matrix in the streaming model , 2013, SODA.

[56]  Michel Deza,et al.  Geometry of cuts and metrics , 2009, Algorithms and combinatorics.

[57]  Daniel M. Kane,et al.  Recent Advances in Algorithmic High-Dimensional Robust Statistics , 2019, ArXiv.

[58]  Maria-Florina Balcan,et al.  Noise-Tolerant Life-Long Matrix Completion via Adaptive Sampling , 2016, NIPS.

[59]  David P. Woodruff,et al.  On Sketching Quadratic Forms , 2015, ITCS.

[60]  David P. Woodruff,et al.  Towards Optimal Moment Estimation in Streaming and Distributed Models , 2019, APPROX-RANDOM.

[61]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[62]  Zhao Song,et al.  Hyperbolic Polynomials I : Concentration and Discrepancy , 2020, ArXiv.

[63]  Haesun Park,et al.  Bounded Matrix Low Rank Approximation , 2012, 2012 IEEE 12th International Conference on Data Mining.

[64]  Shang-Hua Teng,et al.  Spectral Sparsification of Graphs , 2008, SIAM J. Comput..

[65]  Jess Banks,et al.  Pseudospectral Shattering, the Sign Function, and Diagonalization in Nearly Matrix Multiplication Time , 2019, 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).

[66]  Piotr Indyk,et al.  Stable distributions, pseudorandom generators, embeddings, and data stream computation , 2006, JACM.

[67]  G. Pisier,et al.  Non commutative Khintchine and Paley inequalities , 1991 .

[68]  David P. Woodruff,et al.  Matrix Norms in Data Streams: Faster, Multi-Pass and Row-Order , 2016, ICML.

[69]  I. J. Schoenberg Remarks to Maurice Frechet's Article ``Sur La Definition Axiomatique D'Une Classe D'Espace Distances Vectoriellement Applicable Sur L'Espace De Hilbert , 1935 .

[70]  Sanjeev Arora,et al.  Fast algorithms for approximate semidefinite programming using the multiplicative weights update method , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[71]  Mark Rudelson,et al.  Sampling from large matrices: An approach through geometric functional analysis , 2005, JACM.

[72]  M. Maeda,et al.  [Heat conduction]. , 1972, Kango kyoshitsu. [Nursing classroom].

[73]  S. Strogatz Stability, instability and chaos: An introduction to the theory of nonlinear differential equations , 1996 .

[74]  P. Glendinning Stability, Instability and Chaos: An Introduction to the Theory of Nonlinear Differential Equations , 1994 .

[75]  Michael P. Knapp Sines and Cosines of Angles in Arithmetic Progression , 2009 .

[76]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[77]  J. Adell,et al.  Exact Kolmogorov and total variation distances between some familiar discrete distributions , 2006 .

[78]  Gilles Pisier,et al.  Remarks on the non-commutative Khintchine inequalities for 0 , 2008, 0810.5705.

[79]  D. Spielman,et al.  Interlacing Families II: Mixed Characteristic Polynomials and the Kadison-Singer Problem , 2013, 1306.3969.

[80]  Zhao Song,et al.  A Matrix Chernoff Bound for Strongly Rayleigh Distributions and Spectral Sparsifiers from a few Random Spanning Trees , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[81]  Zeyuan Allen Zhu,et al.  A simple, combinatorial algorithm for solving SDD systems in nearly-linear time , 2013, STOC '13.

[82]  Maria-Florina Balcan,et al.  Learning submodular functions , 2010, ECML/PKDD.

[83]  David P. Woodruff,et al.  A Framework for Adversarially Robust Streaming Algorithms , 2020, SIGMOD Rec..

[84]  Jinwoo Shin,et al.  Approximating Spectral Sums of Large-Scale Matrices using Stochastic Chebyshev Approximations , 2017, SIAM J. Sci. Comput..