Weak Decoupling, Polynomial Folds and Approximate Optimization over the Sphere

We consider the following basic problem: given an n-variate degree-d homogeneous polynomial f with real coefficients, compute a unit vector x in R{\string^}n that maximizes abs(f(x)). Besides its fundamental nature, this problem arises in diverse contexts ranging from tensor and operator norms to graph expansion to quantum information theory. The homogeneous degree-2 case is efficiently solvable as it corresponds to computing the spectral norm of an associated matrix, but the higher degree case is NP-hard.We give approximation algorithms for this problem that offer a trade-off between the approximation ratio and running time: in n{\string^}O(q) time, we get an approximation within factor (O(n)\,/\,q){\string^}(d/2-1) for arbitrary polynomials, (O(n)\,/\,q){\string^}(d/4-1/2) for polynomials with non-negative coefficients, and (m\,/\,q){\string^}(1/2) for sparse polynomials with m monomials. The approximation guarantees are with respect to the optimum of the level-q sum-of-squares (SoS) SDP relaxation of the problem (though our algorithms do not rely on actually solving the SDP). Known polynomial time algorithms for this problem rely on decoupling lemmas. Such tools are not capable of offering a trade-off like our results as they blow up the number of variables by a factor equal to the degree. We develop new decoupling tools that are more efficient in the number of variables at the expense of less structure in the output polynomials. This enables us to harness the benefits of higher level SoS relaxations. Our decoupling methods also work with folded polynomials, which are polynomials with polynomials as coefficients. This allows us to exploit easy substructures (such as quadratics) by considering them as coefficients in our algorithms. %We complement our algorithmic results with some polynomially large integrality gaps for d-levels of the SoS relaxation. For general polynomials this follows from known results for random polynomials, which yield a gap of Ω(n){\string^}(d/4-1/2). For polynomials with non-negative coefficients, we prove an Ω(n{\string^}(1/6)\,/\, polylogs) gap for the degree-4 case, based on a novel distribution of 4-uniform hypergraphs. We establish an n{\string^}Ω(d) gap for general degree-d, albeit for a slightly weaker (but still very natural) relaxation. Toward this, we give a method to lift a level-4 solution matrix M to a higher level solution, under a mild technical condition on M.From a structural perspective, our work yields worst-case convergence results on the performance of the sum-of-squares hierarchy for polynomial optimization. Despite the popularity of SoS in this context, such results were previously only known for the case of q = Omega(n).

[1]  Jeff Kahn,et al.  Upper tails for triangles , 2010, Random Struct. Algorithms.

[2]  David Steurer,et al.  Sum-of-squares proofs and the quest toward optimal algorithms , 2014, Electron. Colloquium Comput. Complex..

[3]  Pablo A. Parrilo,et al.  A PTAS for the minimization of polynomials of fixed degree over the simplex , 2006, Theor. Comput. Sci..

[4]  M. Laurent Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[5]  R. Varga,et al.  Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem , 1962 .

[6]  S. Janson,et al.  Upper tails for subgraph counts in random graphs , 2004 .

[7]  Pravesh Kothari,et al.  Quantum entanglement, sum of squares, and the log rank conjecture , 2017, Electron. Colloquium Comput. Complex..

[8]  Tengyu Ma,et al.  Decomposing Overcomplete 3rd Order Tensors using Sum-of-Squares Algorithms , 2015, APPROX-RANDOM.

[9]  David Steurer,et al.  Rounding sum-of-squares relaxations , 2013, Electron. Colloquium Comput. Complex..

[10]  Jonathan Shi,et al.  Tensor principal component analysis via sum-of-square proofs , 2015, COLT.

[11]  Venkatesan Guruswami,et al.  Sum-of-Squares Certificates for Maxima of Random Tensors on the Sphere , 2016, APPROX-RANDOM.

[12]  Venkatesan Guruswami,et al.  Certifying Random Polynomials over the Unit Sphere via Sum of Squares Hierarchy , 2016, ArXiv.

[13]  Aram Wettroth Harrow,et al.  Quantum de Finetti Theorems Under Local Measurements with Applications , 2012, STOC '13.

[14]  Marie Frei,et al.  Decoupling From Dependence To Independence , 2016 .

[15]  Prasad Raghavendra,et al.  Strongly refuting random CSPs below the spectral threshold , 2016, STOC.

[16]  David Steurer,et al.  Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method , 2014, STOC.

[17]  Andrea Montanari,et al.  Improved Sum-of-Squares Lower Bounds for Hidden Clique and Hidden Submatrix Problems , 2015, COLT.

[18]  Van H. Vu,et al.  Divide and conquer martingales and the number of triangles in a random graph , 2004, Random Struct. Algorithms.

[19]  Etienne de Klerk,et al.  An alternative proof of a PTAS for fixed-degree polynomial optimization over the simplex , 2015, Math. Program..

[20]  Alan M. Frieze,et al.  A new approach to the planted clique problem , 2008, FSTTCS.

[21]  Santosh S. Vempala,et al.  Random Tensors and Planted Cliques , 2009, APPROX-RANDOM.

[22]  Sourav Chatterjee,et al.  The missing log in large deviations for triangle counts , 2010, Random Struct. Algorithms.

[23]  Yuan Zhou,et al.  Approximability and proof complexity , 2012, SODA.

[24]  J. Håstad Clique is hard to approximate withinn1−ε , 1999 .

[25]  Yufei Zhao,et al.  On the variational problem for upper tails in sparse random graphs , 2014, Random Struct. Algorithms.

[26]  Srinivasan Venkatesh,et al.  On the advantage over a random assignment , 2002, STOC '02.

[27]  Aram Wettroth Harrow,et al.  Estimating operator norms using covering nets , 2015, ArXiv.

[28]  Etienne de Klerk,et al.  Convergence analysis for Lasserre’s measure-based hierarchy of upper bounds for polynomial optimization , 2017, Math. Program..

[29]  Toniann Pitassi,et al.  Proceedings of the forty-fourth annual ACM symposium on Theory of computing , 2012, STOC 2012.

[30]  Andrea Montanari,et al.  A statistical model for tensor PCA , 2014, NIPS.

[31]  Etienne de Klerk,et al.  The complexity of optimizing over a simplex, hypercube or sphere: a short survey , 2008, Central Eur. J. Oper. Res..

[32]  Miklós Simonovits,et al.  Deterministic and randomized polynomial-time approximation of radii , 2001 .

[33]  Anthony Man-Cho So,et al.  Deterministic approximation algorithms for sphere constrained homogeneous polynomial optimization problems , 2011, Math. Program..

[34]  Jeff Kahn,et al.  Tight upper tail bounds for cliques , 2011, Random Struct. Algorithms.

[35]  Shuzhong Zhang,et al.  Approximation algorithms for homogeneous polynomial optimization with quadratic constraints , 2010, Math. Program..

[36]  Lars Engebretsen,et al.  Clique Is Hard To Approximate Within , 2000 .

[37]  Y. Nesterov Random walk in a simplex and quadratic optimization over convex polytopes , 2003 .

[38]  Subhash Khot,et al.  Linear Equations Modulo 2 and the L1 Diameter of Convex Bodies , 2008, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[39]  Van H. Vu,et al.  A Large Deviation Result on the Number of Small Subgraphs of a Random Graph , 2001, Combinatorics, Probability and Computing.

[40]  Pravesh Kothari,et al.  A Nearly Tight Sum-of-Squares Lower Bound for the Planted Clique Problem , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[41]  J. Lasserre Moments, Positive Polynomials And Their Applications , 2009 .

[42]  Stephanie Wehner,et al.  Convergence of SDP hierarchies for polynomial optimization on the hypersphere , 2012, ArXiv.

[43]  Yuan Zhou,et al.  Hypercontractivity, sum-of-squares proofs, and their applications , 2012, STOC '12.

[44]  Prasad Raghavendra,et al.  The Power of Sum-of-Squares for Detecting Hidden Structures , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[45]  Ryan O'Donnell,et al.  Sum of squares lower bounds for refuting any CSP , 2017, STOC.

[46]  Joan Feigenbaum,et al.  Proceedings of the forty-fifth annual ACM symposium on Theory of computing , 2013, STOC 2013.

[47]  Venkatesan Guruswami,et al.  Multiplicative Approximations for Polynomial Optimization Over the Unit Sphere , 2016, Electron. Colloquium Comput. Complex..