Positive spectrahedra: invariance principles and pseudorandom generators

In a recent work, O’Donnell, Servedio and Tan (STOC 2019) gave explicit pseudorandom generators (PRGs) for arbitrary m-facet polytopes in n variables with seed length poly-logarithmic in m,n, concluding a sequence of works in the last decade, that was started by Diakonikolas, Gopalan, Jaiswal, Servedio, Viola (SICOMP 2010) and Meka, Zuckerman (SICOMP 2013) for fooling linear and polynomial threshold functions, respectively. In this work, we consider a natural extension of PRGs for intersections of positive spectrahedra. A positive spectrahedron is a Boolean function f(x) = [x1A 1 + · · · + xnAn B] where the As are k × k positive semidefinite matrices. We construct explicit PRGs that δ-fool “regular” width-M positive spectrahedra (i.e., when none of the As are dominant) over the Boolean space with seed length poly(log k, logn,M, 1/δ). Our main technical contributions are the following: We first prove an invariance principle for positive spectrahedra via the well-known Lindeberg method. As far as we are aware such a generalization of the Lindeberg method was unknown. Second, we prove an upper bound on noise sensitivity and a Littlewood-Offord theorem for positive spectrahedra. Using these results, we give applications for constructing PRGs for positive spectrahedra, learning theory, discrepancy sets for positive spectrahedra (over the Boolean cube) and PRGs for intersections of structured polynomial threshold functions.

[1]  Sanjeev Arora,et al.  A combinatorial, primal-dual approach to semidefinite programs , 2007, STOC '07.

[2]  P. Erdös On a lemma of Littlewood and Offord , 1945 .

[3]  J. Helton,et al.  NONCOMMUTATIVE CONVEXITY ARISES FROM LINEAR MATRIX INEQUALITIES. , 2006 .

[4]  R. Cooke Real and Complex Analysis , 2011 .

[5]  Paul Tseng,et al.  Analysis of Nonsmooth Symmetric-Matrix-Valued Functions with Applications to Semidefinite Complementarity Problems , 2003, SIAM J. Optim..

[6]  A. Ambrosetti,et al.  A primer of nonlinear analysis , 1993 .

[7]  Yuta Koike,et al.  High-dimensional central limit theorems by Stein’s method , 2020, The Annals of Applied Probability.

[8]  Hristo S. Sendov The higher-order derivatives of spectral functions☆ , 2007 .

[9]  Rocco A. Servedio,et al.  Fooling Gaussian PTFs via local hyperconcentration , 2020, STOC.

[10]  Y. Peres Noise Stability of Weighted Majority , 2004, math/0412377.

[11]  Penghui Yao,et al.  A doubly exponential upper bound on noisy EPR states for binary games , 2019, ArXiv.

[12]  Daniel M. Kane A Pseudorandom Generator for Polynomial Threshold Functions of Gaussian with Subpolynomial Seed Length , 2014, 2014 IEEE 29th Conference on Computational Complexity (CCC).

[13]  Sanjeev Arora,et al.  Fast algorithms for approximate semidefinite programming using the multiplicative weights update method , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[14]  Rocco A. Servedio,et al.  Fooling Intersections of Low-Weight Halfspaces , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[15]  Rocco A. Servedio,et al.  Bounded Independence Fools Halfspaces , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[16]  Yin Tat Lee,et al.  Positive semidefinite programming: mixed, parallel, and width-independent , 2020, STOC.

[17]  Zoltán Füredi,et al.  Solution of the Littlewood-Offord problem in high dimensions , 1988 .

[18]  Pravesh Kothari,et al.  Almost Optimal Pseudorandom Generators for Spherical Caps , 2014, ArXiv.

[19]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[20]  Yin Tat Lee,et al.  Using Optimization to Obtain a Width-Independent, Parallel, Simpler, and Faster Positive SDP Solver , 2015, SODA.

[21]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[22]  V. Bentkus,et al.  Smooth approximations of the norm and differentiable functions with bounded support in banach spacel∞k , 1990 .

[23]  Rahul Jain,et al.  QIP = PSPACE , 2011, JACM.

[24]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[25]  Daniel M. Kane A Small PRG for Polynomial Threshold Functions of Gaussians , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[26]  Daniel M. Kane The Gaussian Surface Area and Noise Sensitivity of Degree-d Polynomial Threshold Functions , 2010, 2010 IEEE 25th Annual Conference on Computational Complexity.

[27]  Louay Bazzi,et al.  Polylogarithmic Independence Can Fool DNF Formulas , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[28]  Rocco A. Servedio,et al.  Agnostically learning halfspaces , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[29]  Daniel M. Kane The average sensitivity of an intersection of half spaces , 2014, STOC.

[30]  Moni Naor,et al.  Small-bias probability spaces: efficient constructions and applications , 1990, STOC '90.

[31]  Prasad Raghavendra,et al.  Lower Bounds on the Size of Semidefinite Programming Relaxations , 2014, STOC.

[32]  Daniel M. Kane,et al.  Pseudorandomness via the Discrete Fourier Transform , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[33]  Ronan Quarez Symmetric determinantal representation of polynomials , 2012 .

[34]  Michael I. Jordan,et al.  Matrix concentration inequalities via the method of exchangeable pairs , 2012, 1201.6002.

[35]  J. Tropp The Expected Norm of a Sum of Independent Random Matrices: An Elementary Approach , 2015, 1506.04711.

[36]  Pravesh Kothari,et al.  A Nearly Tight Sum-of-Squares Lower Bound for the Planted Clique Problem , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[37]  Rahul Jain,et al.  A Parallel Approximation Algorithm for Positive Semidefinite Programming , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[38]  Prahladh Harsha,et al.  An invariance principle for polytopes , 2009, JACM.

[39]  A new derivation of a formula by Kato , 2012 .

[40]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[41]  W. Feller,et al.  An Introduction to Probability Theory and Its Applications, Vol. 1 , 1967 .

[42]  Prasad Raghavendra,et al.  Average Sensitivity and Noise Sensitivity of Polynomial Threshold Functions , 2009, SIAM J. Comput..

[43]  J. Littlewood,et al.  On the number of real roots of a random algebraic equation. II , 1939 .

[44]  J. Littlewood,et al.  On the Number of Real Roots of a Random Algebraic Equation , 1938 .

[45]  Elchanan Mossel,et al.  Maximally stable Gaussian partitions with discrete applications , 2009, 0903.3362.

[46]  J. Lindeberg Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung , 1922 .

[47]  Hristo S. Sendov,et al.  Derivatives of compound matrix valued functions , 2016 .

[48]  Daniel M. Kane,et al.  k-Independent Gaussians Fool Polynomial Threshold Functions , 2010, 2011 IEEE 26th Annual Conference on Computational Complexity.

[49]  R. Bhatia,et al.  Differentiation of Operator Functions and Perturbation Bounds , 1998 .

[50]  Alessandro Panconesi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms , 2009 .

[51]  Rodney Coleman,et al.  Calculus on Normed Vector Spaces , 2012 .

[52]  Hristo S. Sendov,et al.  Asymptotic expansions of the ordered spectrum of symmetric matrices , 2010 .

[53]  Rocco A. Servedio,et al.  Every Linear Threshold Function has a Low-Weight Approximator , 2006, 21st Annual IEEE Conference on Computational Complexity (CCC'06).

[54]  Rajendra Bhatia,et al.  Derivations, derivatives and chain rules , 1999 .

[55]  Ryan O'Donnell,et al.  Noise stability of functions with low influences: Invariance and optimality , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[56]  A. S. Lewis,et al.  Derivatives of Spectral Functions , 1996, Math. Oper. Res..

[57]  Zhi-Quan Luo,et al.  Matrix convex functions with applications to weighted centers for semidefinite programming , 2005 .

[58]  David Zuckerman,et al.  Pseudorandom generators for polynomial threshold functions , 2009, STOC '10.

[59]  Richard Peng,et al.  Faster and simpler width-independent parallel algorithms for positive semidefinite programming , 2012, SPAA '12.

[60]  Bernd Grtner,et al.  Approximation Algorithms and Semidefinite Programming , 2012 .

[61]  Hans Raj Tiwary,et al.  Exponential Lower Bounds for Polytopes in Combinatorial Optimization , 2011, J. ACM.

[62]  Xiaodi Wu,et al.  Parallel Approximation of Min-Max Problems , 2010, computational complexity.

[63]  Terence Tao,et al.  The Littlewood-Offord problem in high dimensions and a conjecture of Frankl and Füredi , 2010, Comb..

[64]  Rocco A. Servedio,et al.  Fooling polytopes , 2018, Electron. Colloquium Comput. Complex..

[65]  Ryan O'Donnell,et al.  Learning Geometric Concepts via Gaussian Surface Area , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[66]  Rocco A. Servedio,et al.  Simple and efficient pseudorandom generators from gaussian processes , 2019, Electron. Colloquium Comput. Complex..

[67]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[68]  P. Gopalan,et al.  Fooling Functions of Halfspaces under Product Distributions , 2010, 2010 IEEE 25th Annual Conference on Computational Complexity.

[69]  R. Schapire,et al.  Toward efficient agnostic learning , 1992, COLT '92.

[70]  P. Parrilo,et al.  Semidefinite Representation of the k-Ellipse , 2007, math/0702005.

[71]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[72]  Rekha R. Thomas,et al.  Semidefinite Optimization and Convex Algebraic Geometry , 2012 .

[73]  Rajendra Bhatia,et al.  Pinching, Trimming, Truncating, and Averaging of Matrices , 2000, Am. Math. Mon..

[74]  Rahul Jain,et al.  Two-Message Quantum Interactive Proofs Are in PSPACE , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[75]  T. Sanders,et al.  Analysis of Boolean Functions , 2012, ArXiv.

[76]  Elchanan Mossel,et al.  Gaussian Bounds for Noise Correlation of Functions and Tight Analysis of Long Codes , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.