Semialgebraic Proofs and Efficient Algorithm Design

Over the last twenty years, an exciting interplay has emerged between proof systems and algorithms. Some natural families of algorithms can be viewed as a generic translation from a proof that a solution exists into an algorithm for finding the solution itself. This connection has perhaps been the most consequential in the context of semi-algebraic proof systems and basic primitives in algorithm design such as linear and semidefinite programming. The proof system perspective, in this context, has provided fundamentally new tools for both algorithm design and analysis. These news tools have helped in both designing better algorithms for well-studied problems and proving tight lower bounds on such techniques. This monograph is aimed at expositing this interplay between proof systems and efficient algorithm design and surveying the state-of-the-art for two of the most important semi-algebraic proof systems: Sherali-Adams and Sum-ofSquares. We rigorously develop and survey the state-of-the-art for Sherali-Adams and Sum-of-Squares both as proof systems, ∗Research supported by NSERC. Noah Fleming, Pravesh Kothari and Toniann Pitassi (2019), “Semialgebraic Proofs and Efficient Algorithm Design”, Foundations and Trends © in Theoretical Computer Science: Vol. 14, No. 1–2, pp 1–221. DOI: 10.1561/0400000086. Full text available at: http://dx.doi.org/10.1561/0400000086

[1]  Prasad Raghavendra,et al.  The Power of Sum-of-Squares for Detecting Hidden Structures , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[2]  Avi Wigderson,et al.  On span programs , 1993, [1993] Proceedings of the Eigth Annual Structure in Complexity Theory Conference.

[3]  Madhur Tulsiani,et al.  SDP Gaps from Pairwise Independence , 2012, Theory Comput..

[4]  Ryan O'Donnell,et al.  SOS Is Not Obviously Automatizable, Even Approximately , 2016, ITCS.

[5]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[6]  Prasad Raghavendra,et al.  On the Bit Complexity of Sum-of-Squares Proofs , 2017, ICALP.

[7]  Vasek Chvátal,et al.  Determining the Stability Number of a Graph , 1976, SIAM J. Comput..

[8]  Pravesh Kothari,et al.  Sum-of-Squares Meets Program Obfuscation, Revisited , 2019, IACR Cryptol. ePrint Arch..

[9]  Prasad Raghavendra,et al.  Rounding Semidefinite Programming Hierarchies via Global Correlation , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[10]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[11]  Wenceslas Fernandez de la Vega,et al.  Linear programming relaxations of maxcut , 2007, SODA '07.

[12]  Hanif D. Sherali,et al.  A Hierarchy of Relaxations and Convex Hull Characterizations for Mixed-integer Zero-one Programming Problems , 1994, Discret. Appl. Math..

[13]  Barnaby Martin,et al.  Tight rank lower bounds for the Sherali-Adams proof system , 2009, Theor. Comput. Sci..

[14]  G. Rozenber,et al.  Current Trends in Theoretical Computer Science , 1993 .

[15]  Ran Raz,et al.  On Interpolation and Automatization for Frege Systems , 2000, SIAM J. Comput..

[16]  Shachar Lovett,et al.  Rectangles Are Nonnegative Juntas , 2015, SIAM J. Comput..

[17]  Grant Schoenebeck,et al.  Linear Level Lasserre Lower Bounds for Certain k-CSPs , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[18]  Sanjeev Arora,et al.  Expander flows, geometric embeddings and graph partitioning , 2009, JACM.

[19]  Ankit Garg,et al.  Monotone circuit lower bounds from resolution , 2018, Electron. Colloquium Comput. Complex..

[20]  Christoph Berkholz,et al.  The Relation between Polynomial Calculus, Sherali-Adams, and Sum-of-Squares Proofs , 2017, Electron. Colloquium Comput. Complex..

[21]  Jan Krajícek,et al.  Lower bounds on Hilbert's Nullstellensatz and propositional proofs , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[22]  Aravindan Vijayaraghavan,et al.  On Learning Mixtures of Well-Separated Gaussians , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[23]  J. Krivine,et al.  Anneaux préordonnés , 1964 .

[24]  Yuan Zhou,et al.  Hypercontractivity, sum-of-squares proofs, and their applications , 2012, STOC '12.

[25]  Pravesh Kothari,et al.  Sum of Squares Lower Bounds from Pairwise Independence , 2015, STOC.

[26]  Prasad Raghavendra,et al.  On the Integrality Gap of Degree-4 Sum of Squares for Planted Clique , 2016, SODA.

[27]  David P. Williamson,et al.  Improved approximation algorithms for MAX SAT , 2000, SODA '00.

[28]  David Steurer,et al.  Exact tensor completion with sum-of-squares , 2017, COLT.

[29]  Subhash Khot.  On the Unique Games Conjecture (Invited Survey) , 2010, Computational Complexity Conference.

[30]  N. Z. Shor.  An approach to obtaining global extremums in polynomial mathematical programming problems , 1987 .

[31]  Ran Raz,et al.  Separation of the Monotone NC Hierarchy , 1999, Comb..

[32]  John Dunagan,et al.  A Polynomial-time Rescaling Algorithm for Solving Linear Programs , 2003 .

[33]  Uri Zwick,et al.  Improved Rounding Techniques for the MAX 2-SAT and MAX DI-CUT Problems , 2002, IPCO.

[34]  David Steurer,et al.  Sum-of-squares proofs and the quest toward optimal algorithms , 2014, Electron. Colloquium Comput. Complex..

[35]  Pravesh Kothari,et al.  Efficient Algorithms for Outlier-Robust Regression , 2018, COLT.

[36]  Pravesh Kothari,et al.  A Nearly Tight Sum-of-Squares Lower Bound for the Planted Clique Problem , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[37]  Toniann Pitassi,et al.  Deterministic Communication vs. Partition Number , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[38]  David Steurer,et al.  Rounding sum-of-squares relaxations , 2013, Electron. Colloquium Comput. Complex..

[39]  Henry Wolkowicz,et al.  Strong Duality for Semidefinite Programming , 1997, SIAM J. Optim..

[40]  Toniann Pitassi,et al.  Lifting Nullstellensatz to monotone span programs over any field , 2018, Electron. Colloquium Comput. Complex..

[41]  Béla Bollobás,et al.  Proving integrality gaps without knowing the linear program , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[42]  Russell Impagliazzo,et al.  Homogenization and the polynomial calculus , 2000, computational complexity.

[43]  Noah Fleming.  Approximate Constraint Satisfaction Requires Large LP Relaxations , 2018 .

[44]  D. J. Newman,et al.  Arithmetic, Geometric Inequality , 1960 .

[45]  Toniann Pitassi,et al.  Integrality Gaps of 2-o(1) for Vertex Cover SDPs in the Lov[a-acute]sz--Schrijver Hierarchy , 2010, SIAM J. Comput..

[46]  Guy Kindler,et al.  Optimal inapproximability results for MAX-CUT and other 2-variable CSPs? , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[47]  Eli Ben-Sasson,et al.  Short proofs are narrow—resolution made simple , 2001, JACM.

[48]  L. Trevisan.  On Khot’s unique games conjecture , 2012 .

[49]  Pravesh Kothari,et al.  Quantum entanglement, sum of squares, and the log rank conjecture , 2017, Electron. Colloquium Comput. Complex..

[50]  Hilary Putnam,et al.  A Computing Procedure for Quantification Theory , 1960, JACM.

[51]  Nathan Segerlind,et al.  The Complexity of Propositional Proofs , 2007, Bull. Symb. Log..

[52]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[53]  Dima Grigoriev,et al.  Tseitin's tautologies and lower bounds for Nullstellensatz proofs , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[54]  Sanjeev Arora,et al.  Probabilistic checking of proofs; a new characterization of NP , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[55]  Ankur Moitra,et al.  Noisy tensor completion via the sum-of-squares hierarchy , 2015, Mathematical Programming.

[56]  Dima Grigoriev,et al.  Complexity of Null-and Positivstellensatz proofs , 2001, Ann. Pure Appl. Log..

[57]  Toniann Pitassi,et al.  Query-to-Communication Lifting for BPP , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[58]  L. Lovász,et al.  Geometric Algorithms and Combinatorial Optimization , 1981 .

[59]  Olga Taussky-Todd.  SOME CONCRETE ASPECTS OF HILBERT'S 17TH PROBLEM , 1996 .

[60]  Pravesh Kothari,et al.  Limits on Low-Degree Pseudorandom Generators (Or: Sum-of-Squares Meets Program Obfuscation) , 2018, Electron. Colloquium Comput. Complex..

[61]  Ryan O'Donnell,et al.  Sum of squares lower bounds for refuting any CSP , 2017, STOC.

[62]  David Steurer,et al.  Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method , 2014, STOC.

[63]  László Lovász,et al.  On the Shannon capacity of a graph , 1979, IEEE Trans. Inf. Theory.

[64]  Toniann Pitassi,et al.  Simplified and improved resolution lower bounds , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[65]  Alexander A. Sherstov.  The Pattern Matrix Method , 2009, SIAM J. Comput..

[66]  Toniann Pitassi,et al.  Algebraic Propositional Proof Systems , 1996, Descriptive Complexity and Finite Models.

[67]  Pravesh Kothari,et al.  Better Agnostic Clustering Via Relaxed Tensor Norms , 2017, ArXiv.

[68]  Motakuri V. Ramana,et al.  An exact duality theory for semidefinite programming and its complexity implications , 1997, Math. Program..

[69]  Jakob Nordström.  On the interplay between proof complexity and SAT solving , 2015, SIGL.

[70]  Massimo Lauria,et al.  On the Automatizability of Polynomial Calculus , 2010, Theory of Computing Systems.

[71]  Teofilo F. Gonzalez,et al.  P-Complete Approximation Problems , 1976, J. ACM.

[72]  G. Stengle.  A nullstellensatz and a positivstellensatz in semialgebraic geometry , 1974 .

[73]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[74]  Konstantinos Georgiou,et al.  Better Balance by Being Biased , 2016, ACM Trans. Algorithms.

[75]  Prasad Raghavendra,et al.  Optimal algorithms and inapproximability results for every CSP? , 2008, STOC.

[76]  N. Z. Shor.  Cut-off method with space extension in convex programming problems , 1977, Cybernetics.

[77]  Stephen A. Cook,et al.  The Relative Efficiency of Propositional Proof Systems , 1979, Journal of Symbolic Logic.

[78]  Fu Li,et al.  Non-Commutative Formulas and Frege Lower Bounds: a New Characterization of Propositional Proofs , 2015, Computational Complexity Conference.

[79]  Michael Alekhnovich,et al.  Resolution Is Not Automatizable Unless W[P] Is Tractable , 2008, SIAM J. Comput..

[80]  Samuel R. Buss,et al.  Good degree bounds on Nullstellensatz refutations of the induction principle , 1996, Proceedings of Computational Complexity (Formerly Structure in Complexity Theory).

[81]  Alexander A. Razborov.  Guest Column: Proof Complexity and Beyond , 2016, SIGA.

[82]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[83]  Miklós Ajtai,et al.  The complexity of the Pigeonhole Principle , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[84]  Jerry Li,et al.  Mixture models, robustness, and sum of squares proofs , 2017, STOC.

[85]  Jan Krajícek,et al.  Some Consequences of Cryptographical Conjectures for S12 and EF , 1998, Inf. Comput..

[86]  David P. Williamson,et al.  .879-approximation algorithms for MAX CUT and MAX 2SAT , 1994, STOC '94.

[87]  Jan Krajícek,et al.  Exponential Lower Bounds for the Pigeonhole Principle , 1992, STOC.

[88]  T. Rothvoss.  The Lasserre hierarchy in Approximation algorithms Lecture Notes for the MAPSP 2013 Tutorial Preliminary version , 2013 .

[89]  Claire Mathieu,et al.  Sherali-adams relaxations of the matching polytope , 2009, STOC '09.

[90]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[91]  Claus Scheiderer,et al.  Sums of squares on real algebraic curves , 2003 .

[92]  Madhur Tulsiani.  CSP gaps and reductions in the lasserre hierarchy , 2009, STOC '09.

[93]  Moses Charikar,et al.  Near-optimal algorithms for unique games , 2006, STOC '06.

[94]  Toniann Pitassi,et al.  Exponential Lower Bounds and Integrality Gaps for Tree-Like Lovász-Schrijver Procedures , 2009, SIAM J. Comput..

[95]  Michel X. Goemans,et al.  Semideenite Programming in Combinatorial Optimization , 1999 .

[96]  Avi Wigderson,et al.  Proof Complexity Lower Bounds from Algebraic Circuit Complexity , 2016, Electron. Colloquium Comput. Complex..

[97]  Sanjeev Arora,et al.  Subexponential Algorithms for Unique Games and Related Problems , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[98]  Carsten Lund,et al.  Proof verification and the hardness of approximation problems , 1998, JACM.

[99]  Venkatesan Guruswami,et al.  Clustering with qualitative information , 2005, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[100]  Subhash Khot.  On the power of unique 2-prover 1-round games , 2002, STOC '02.

[101]  Pablo A. Parrilo,et al.  Semidefinite programming relaxations for semialgebraic problems , 2003, Math. Program..

[102]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[103]  László Lovász,et al.  Semidefinite Programs and Combinatorial Optimization , 2003 .

[104]  Armin Haken,et al.  The Intractability of Resolution , 1985, Theor. Comput. Sci..

[105]  Didier Henrion,et al.  Strong duality in Lasserre’s hierarchy for polynomial optimization , 2014, Optim. Lett..

[106]  Toniann Pitassi,et al.  Circuit Complexity, Proof Complexity, and Polynomial Identity Testing , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[107]  Nathan Linial.  Finite metric spaces: combinatorics, geometry and algorithms , 2002, SCG '02.

[108]  Yuval Rabani,et al.  On the Hardness of Approximating Multicut and Sparsest-Cut , 2005, Computational Complexity Conference.

[109]  Tengyu Ma,et al.  Polynomial-Time Tensor Decompositions with Sum-of-Squares , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[110]  Russell Impagliazzo,et al.  Lower bounds for the polynomial calculus and the Gröbner basis algorithm , 1999, computational complexity.

[111]  Konstantin Makarychev,et al.  How to Play Unique Games Using Embeddings , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[112]  Dima Grigoriev,et al.  Linear lower bound on degrees of Positivstellensatz calculus proofs for the parity , 2001, Theor. Comput. Sci..

[113]  Toniann Pitassi,et al.  Non-Automatizability of Bounded-Depth Frege Proofs , 2004, computational complexity.

[114]  Subhash Khot,et al.  Vertex cover might be hard to approximate to within 2-/spl epsiv/ , 2003, 18th IEEE Annual Conference on Computational Complexity, 2003. Proceedings..

[115]  Noga Alon,et al.  Approximating the cut-norm via Grothendieck's inequality , 2004, STOC '04.

[116]  Avi Wigderson,et al.  Sum-of-squares Lower Bounds for Planted Clique , 2015, STOC.

[117]  Mihir Bellare,et al.  Free Bits, PCPs, and Nonapproximability-Towards Tight Results , 1998, SIAM J. Comput..

[118]  Dima Grigoriev,et al.  Algebraic proof systems over formulas , 2003, Electron. Colloquium Comput. Complex..

[119]  Johan Håstad,et al.  Some optimal inapproximability results , 2001, JACM.

[120]  Subhash Khot,et al.  Pseudorandom Sets in Grassmann Graph Have Near-Perfect Expansion , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[121]  Jean B. Lasserre,et al.  An Explicit Exact SDP Relaxation for Nonlinear 0-1 Programs , 2001, IPCO.

[122]  Alasdair Urquhart,et al.  Formal Languages]: Mathematical Logic--mechanical theorem proving , 2022 .

[123]  Pravesh Kothari,et al.  Outlier-robust moment-estimation via sum-of-squares , 2017, ArXiv.

[124]  Endre Szemerédi,et al.  Many hard examples for resolution , 1988, JACM.

[125]  Venkatesan Guruswami,et al.  Lasserre Hierarchy, Higher Eigenvalues, and Approximation Schemes for Graph Partitioning and Quadratic Integer Programming with PSD Objectives , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[126]  E. Artin.  Über die Zerlegung definiter Funktionen in Quadrate , 1927 .

[127]  Prasad Raghavendra,et al.  Approximating rectangles by juntas and weakly-exponential lower bounds for LP relaxations of CSPs , 2016, STOC.

[128]  Russell Impagliazzo,et al.  Using the Groebner basis algorithm to find proofs of unsatisfiability , 1996, STOC '96.

[129]  Prasad Raghavendra,et al.  Graph expansion and the unique games conjecture , 2010, STOC '10.

[130]  Moses Charikar,et al.  Integrality gaps for Sherali-Adams relaxations , 2009, STOC '09.

[131]  Samuel R. Buss,et al.  Linear gaps between degrees for the polynomial calculus modulo distinct primes , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[132]  Alexander A. Razborov,et al.  Lower bounds for the polynomial calculus , 1998, computational complexity.

[133]  Albert Atserias,et al.  Size-degree trade-offs for sums-of-squares and positivstellensatz proofs , 2018, CCC.

[134]  Nisheeth K. Vishnoi,et al.  The Unique Games Conjecture, Integrality Gap for Cut Problems and Embeddability of Negative Type Metrics into l1 , 2005, FOCS.

[135]  Donald W. Loveland,et al.  A machine program for theorem-proving , 2011, CACM.

[136]  Albert Atserias,et al.  Automating Resolution is NP-Hard , 2019, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS).