Sums of Squares, Moment Matrices and Optimization Over Polynomials

We consider the problem of minimizing a polynomial over a semialgebraic set defined by polynomial equations and inequalities, which is NP-hard in general. Hierarchies of semidefinite relaxations have been proposed in the literature, involving positive semidefinite moment matrices and the dual theory of sums of squares of polynomials. We present these hierarchies of approximations and their main properties: asymptotic/finite convergence, optimality certificate, and extraction of global optimum solutions. We review the mathematical tools underlying these properties, in particular, some sums of squares representation results for positive polynomials, some results about moment matrices (in particular, of Curto and Fialkow), and the algebraic eigenvalue method for solving zero-dimensional systems of polynomial equations. We try whenever possible to provide detailed proofs and background.

[1]  D. Hilbert.  Über die Darstellung definiter Formen als Summe von Formenquadraten , 1888 .

[2]  E. Haviland,et al.  On the Momentum Problem for Distribution Functions in More Than One Dimension. II , 1935 .

[3]  H. Whitney.  Elementary Structure of Real Algebraic Varieties , 1957 .

[4]  R. Bellman,et al.  Proceedings of Symposia in Applied Mathematics. , 1961 .

[5]  J. Krivine,et al.  Anneaux préordonnés , 1964 .

[6]  T. Motzkin,et al.  Maxima for Graphs and a New Proof of a Theorem of Turán , 1965, Canadian Journal of Mathematics.

[7]  A. Nussbaum.  Quasi-analytic vectors , 1965 .

[8]  N. Akhiezer,et al.  The Classical Moment Problem. , 1968 .

[9]  G. Stengle.  A nullstellensatz and a positivstellensatz in semialgebraic geometry , 1974 .

[10]  Christian Berg,et al.  Positive definite functions on Abelian semigroups , 1976 .

[11]  Man-Duen Choi,et al.  Extremal positive semidefinite forms , 1977 .

[12]  M. Kreĭn,et al.  The Markov Moment Problem and Extremal Problems , 1977 .

[13]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[14]  B. Reznick.  Extremal PSD forms with few terms , 1978 .

[15]  Christian Berg,et al.  A remark on the multidimensional moment problem , 1979 .

[16]  László Lovász,et al.  On the Shannon capacity of a graph , 1979, IEEE Trans. Inf. Theory.

[17]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[18]  Bruce Reznick,et al.  Real zeros of positive semidefinite forms. I , 1980 .

[19]  Charles R. Johnson,et al.  Positive definite completions of partial Hermitian matrices , 1984 .

[20]  Christian Berg,et al.  Exponentially bounded positive definite functions , 1984 .

[21]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[22]  N. Z. Shor.  Class of global minimum bounds of polynomial functions , 1987 .

[23]  N. Z. Shor.  An approach to obtaining global extremums in polynomial mathematical programming problems , 1987 .

[24]  H. Landau.  Moments in mathematics , 1987 .

[25]  M-F Roy,et al.  Géométrie algébrique réelle , 1987 .

[26]  C. Berg.  The multidimensional moment problem and semi-groups , 1987 .

[27]  D. S. Arnon,et al.  Algorithms in real algebraic geometry , 1988 .

[28]  D. Handelman.  Representing polynomials by positive linear functions on compact convex polyhedra. , 1988 .

[29]  J. G. Pierce,et al.  Geometric Algorithms and Combinatorial Optimization , 2016 .

[30]  Warren P. Adams,et al.  A hierarchy of relaxation between the continuous and convex hull representations , 1990 .

[31]  Hanif D. Sherali,et al.  A Hierarchy of Relaxations Between the Continuous and Convex Hull Representations for Zero-One Programming Problems , 1990, SIAM J. Discret. Math..

[32]  K. Schmüdgen.  TheK-moment problem for compact semi-algebraic sets , 1991 .

[33]  K. Schmüdgen.  TheK-moment problem for compact semi-algebraic sets , 1991 .

[34]  Alexander Schrijver,et al.  Cones of Matrices and Set-Functions and 0-1 Optimization , 1991, SIAM J. Optim..

[35]  Panos M. Pardalos,et al.  Open questions in complexity theory for numerical optimization , 1992, Mathematical programming.

[36]  Marie-Françoise Roy,et al.  Real algebraic geometry , 1992 .

[37]  B. Peyton,et al.  An Introduction to Chordal Graphs and Clique Trees , 1993 .

[38]  Dinesh Manocha,et al.  SOLVING SYSTEMS OF POLYNOMIAL EQUATIONS , 2002 .

[39]  B. Reznick,et al.  Sums of squares of real polynomials , 1995 .

[40]  B. Reznick.  Uniform denominators in Hilbert's seventeenth problem , 1995 .

[41]  H. M. Möller,et al.  Multivariate polynomial equations with multiple zeros solved by matrix eigenproblems , 1995 .

[42]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[43]  Olga Taussky-Todd.  SOME CONCRETE ASPECTS OF HILBERT'S 17TH PROBLEM , 1996 .

[44]  Raúl E. Curto,et al.  Solution of the Truncated Complex Moment Problem for Flat Data , 1996 .

[45]  Jan Arne Telle,et al.  Algorithms for Vertex Partitioning Problems on Partial k-Trees , 1997, SIAM J. Discret. Math..

[46]  Donal O'Shea,et al.  Ideals, varieties, and algorithms - an introduction to computational algebraic geometry and commutative algebra (2. ed.) , 1997, Undergraduate texts in mathematics.

[47]  Mihai Putinar,et al.  A note on Tchakaloff’s Theorem , 1997 .

[48]  Masakazu Kojima,et al.  Semidefinite Programming Relaxation for Nonconvex Quadratic Programs , 1997, J. Glob. Optim..

[49]  Robert M. Corless,et al.  A reordered Schur factorization method for zero-dimensional polynomial systems with multiple roots , 1997, ISSAC.

[50]  L. van den Dries,et al.  Tame Topology and O-minimal Structures , 1998 .

[51]  V. Powers,et al.  An algorithm for sums of squares of real polynomials , 1998 .

[52]  R. Curto,et al.  Flat Extensions of Positive Moment Matrices: Recursively Generated Relations , 1998 .

[53]  N. Shor.  Nondifferentiable Optimization and Polynomial Problems , 1998 .

[54]  David A. Cox,et al.  Using Algebraic Geometry , 1998 .

[55]  Arjeh M. Cohen,et al.  Some tapas of computer algebra , 1999, Algorithms and computation in mathematics.

[56]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[57]  Bernard Mourrain,et al.  A New Criterion for Normal Form Algorithms , 1999, AAECC.

[58]  Yurii Nesterov,et al.  Squared Functional Systems and Optimization Problems , 2000 .

[59]  Henry Wolkowicz,et al.  Handbook of Semidefinite Programming , 2000 .

[60]  A. Fialkow,et al.  THE TRUNCATED COMPLEX K-MOMENT PROBLEM , 2000 .

[61]  B. Reznick.  Some concrete aspects of Hilbert's 17th Problem , 2000 .

[62]  Klaus Jansen,et al.  On the Complexity of the Maximum Cut Problem , 1994, Nord. J. Comput..

[63]  B. Reznick,et al.  Polynomials that are positive on an interval , 2000 .

[64]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[65]  R. Curto,et al.  The truncated complex -moment problem , 2000 .

[66]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[67]  Jan Stochel,et al.  Solving the truncated moment problem solves the full moment problem , 2001, Glasgow Mathematical Journal.

[68]  Charles N. Delzell,et al.  Positive Polynomials: From Hilbert’s 17th Problem to Real Algebra , 2001 .

[69]  A. Prestel,et al.  Distinguished representations of strictly positive polynomials , 2001 .

[70]  B. Reznick,et al.  A new bound for Pólya's Theorem with applications to polynomials positive on polyhedra , 2001 .

[71]  Jean B. Lasserre,et al.  Polynomials nonnegative on a grid and discrete optimization , 2001 .

[72]  Jean B. Lasserre,et al.  An Explicit Exact SDP Relaxation for Nonlinear 0-1 Programs , 2001, IPCO.

[73]  Pablo A. Parrilo,et al.  Minimizing Polynomial Functions , 2001, Algorithmic and Quantitative Aspects of Real Algebraic Geometry in Mathematics and Computer Science.

[74]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[75]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[76]  Victoria Powers,et al.  The moment problem for non-compact semialgebraic sets , 2001 .

[77]  James Renegar,et al.  A mathematical view of interior-point methods in convex optimization , 2001, MPS-SIAM series on optimization.

[78]  P. Parrilo.  An explicit construction of distinguished representations of polynomials nonnegative over finite sets , 2002 .

[79]  Alexander Barvinok,et al.  A course in convexity , 2002, Graduate studies in mathematics.

[80]  Salma Kuhlmann,et al.  Positivity, sums of squares and the multi-dimensional moment problem , 2002 .

[81]  Etienne de Klerk,et al.  Approximation of the Stability Number of a Graph via Copositive Programming , 2002, SIAM J. Optim..

[82]  Jean B. Lasserre,et al.  Semidefinite Programming vs. LP Relaxations for Polynomial Programming , 2002, Math. Oper. Res..

[83]  E. D. Klerk,et al.  Aspects of semidefinite programming : interior point algorithms and selected applications , 2002 .

[84]  Markus Schweighofer,et al.  An algorithmic approach to Schmudgen's Positivstellensatz , 2002 .

[85]  Franz Rendl,et al.  Semidefinite programming and integer programming , 2002 .

[86]  Murray Marshall.  Optimization of Polynomial Functions , 2003, Canadian Mathematical Bulletin.

[87]  Bernard Hanzon,et al.  REPORT RAPPORT , 2001 .

[88]  Grigoriy Blekherman.  There are significantly more nonegative polynomials than sums of squares , 2003, math/0309130.

[89]  Monique Laurent,et al.  A Comparison of the Sherali-Adams, Lovász-Schrijver, and Lasserre Relaxations for 0-1 Programming , 2003, Math. Oper. Res..

[90]  Pablo A. Parrilo,et al.  Semidefinite programming relaxations for semialgebraic problems , 2003, Math. Program..

[91]  J. Lasserre,et al.  Detecting global optimality and extracting solutions in GloptiPoly , 2003 .

[92]  Didier Henrion,et al.  GloptiPoly: Global optimization over polynomials with Matlab and SeDuMi , 2003, TOMS.

[93]  D. Jibetean,et al.  Algebraic Optimization with Applications in System Theory , 2003 .

[94]  D. Kamenetsky.  Symmetry Groups , 2003 .

[95]  P. Parrilo,et al.  Symmetry groups, semidefinite programs, and sums of squares , 2002, math/0211450.

[96]  P. Parrilo,et al.  From coefficients to samples: a new approach to SOS optimization , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[97]  Jean B. Lasserre,et al.  S.o.s. approximation of polynomials nonnegative on a real algebraic set , 2004 .

[98]  R. Freund.  Review of A mathematical view of interior-point methods in convex optimization, by James Renegar, SIAM, Philadelphia, PA , 2004 .

[99]  Frank Sottile,et al.  A new approach to Hilbert's theorem on ternary quartics , 2004 .

[100]  Markus Schweighofer,et al.  On the complexity of Schmu"dgen's Positivstellensatz , 2004, J. Complex..

[101]  Monique Laurent,et al.  Semidefinite Relaxations for Max-Cut , 2004, The Sharpest Cut.

[102]  C. Wampler,et al.  Basic Algebraic Geometry , 2005 .

[103]  Markus Schweighofer,et al.  Optimization of Polynomials on Compact Semialgebraic Sets , 2005, SIAM J. Optim..

[104]  P. Parrilo.  Exploiting Algebraic Structure in Sum of Squares Programs , 2005 .

[105]  Monique Laurent,et al.  Semidefinite bounds for the stability number of a graph via sums of squares of polynomials , 2005, Math. Program..

[106]  Jean B. Lasserre.  Sum of Squares Approximation of Polynomials, Nonnegative on a Real Algebraic Set , 2005, SIAM J. Optim..

[107]  Jean B. Lasserre,et al.  SOS approximations of nonnegative polynomials via simple high degree perturbations , 2005 .

[108]  Jean B. Lasserre,et al.  Polynomial Programming: LP-Relaxations Also Converge , 2005, SIAM J. Optim..

[109]  Alexander Schrijver,et al.  New code upper bounds from the Terwilliger algebra and semidefinite programming , 2005, IEEE Transactions on Information Theory.

[110]  David A. Cox,et al.  Solving Polynomial Equations: Foundations, Algorithms, and Applications (Algorithms and Computation in Mathematics) , 2005 .

[111]  Martin Kreuzer,et al.  An algebraist’s view on border bases , 2005 .

[112]  Josef Teichmann,et al.  The proof of Tchakaloff’s Theorem , 2005 .

[113]  Monique Laurent,et al.  Semidefinite Approximations for Global Unconstrained Polynomial Optimization , 2005, SIAM J. Optim..

[114]  Monique Laurent,et al.  Revisiting two theorems of Curto and Fialkow on moment matrices , 2005 .

[115]  Masakazu Kojima,et al.  Sparsity in sums of squares of polynomials , 2005, Math. Program..

[116]  J. Lasserre,et al.  SOS approximation of polynomials nonnegative on an algebraic set , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[117]  P. Parrilo,et al.  On the equivalence of algebraic approaches to the minimization of forms on the simplex , 2005 .

[118]  Pablo A. Parrilo,et al.  A PTAS for the minimization of polynomials of fixed degree over the simplex , 2006, Theor. Comput. Sci..

[119]  Jean B. Lasserre.  A Sum of Squares Approximation of Nonnegative Polynomials , 2006, SIAM J. Optim..

[120]  Lieven Vandenberghe,et al.  Discrete Transforms, Semidefinite Programming, and Sum-of-Squares Representations of Nonnegative Polynomials , 2006, SIAM J. Optim..

[121]  Dick den Hertog,et al.  Optimization of Univariate Functions on Bounded Intervals by Interpolation and Semidefinite Programming , 2006 .

[122]  P. Rostalski,et al.  Semidefinite characterization and computation of real radical ideals , 2006 .

[123]  H. Michael Möller,et al.  The Extremal Truncated Moment Problem , 2006 .

[124]  Masakazu Muramatsu,et al.  Sums of Squares and Semidefinite Programming Relaxations for Polynomial Optimization Problems with Structured Sparsity , 2004 .

[125]  C. Bachoc,et al.  New upper bounds for kissing numbers from semidefinite programming , 2006, math/0608426.

[126]  Jean B. Lasserre,et al.  Convergent SDP-Relaxations in Polynomial Optimization with Sparsity , 2006, SIAM J. Optim..

[128]  Jean B. Lasserre.  Sufficient conditions for a real polynomial to be a sum of squares , 2006 .

[129]  David Grimm,et al.  A note on the representation of positive polynomials with structured sparsity , 2006, math/0611498.

[130]  Mihai Putinar,et al.  POSITIVE POLYNOMIALS IN SCALAR AND MATRIX VARIABLES, THE SPECTRAL THEOREM AND OPTIMIZATION , 2006 .

[131]  Markus Schweighofer.  Global Optimization of Polynomials Using Gradient Tentacles and Sums of Squares , 2006, SIAM J. Optim..

[132]  James Demmel,et al.  Minimizing Polynomials via Sum of Squares over the Gradient Ideal , 2004, Math. Program..

[133]  Lawrence A. Fialkow.  TRUNCATED MULTIVARIABLE MOMENT PROBLEMS WITH FINITE VARIETY , 2007 .

[134]  Monique Laurent,et al.  Strengthened semidefinite programming bounds for codes , 2007, Math. Program..

[135]  Monique Laurent,et al.  Semidefinite representations for finite varieties , 2007, Math. Program..

[136]  Alexander Schrijver,et al.  Reduction of symmetric semidefinite programs using the regular $$\ast$$-representation , 2007, Math. Program..

[137]  F. Vallentin.  Symmetry in semidefinite programs , 2007, 0706.4233.

[138]  Bruce Reznick,et al.  On Hilbert's construction of positive polynomials , 2007, 0707.2156.

[139]  E. D. Klerk,et al.  Exploiting group symmetry in truss topology optimization , 2007 .

[140]  Exploiting Group Symmetry in Semidefinite Programming Relaxations of the Quadratic Assignment Problem , 2007 .

[141]  D. Henrion.  On semidefinite representations of plane quartics , 2008, 0809.1826.

[142]  Markus Schweighofer,et al.  On the complexity of Putinar's Positivstellensatz , 2005, 0812.2657.

[143]  James Demmel,et al.  Sparse SOS Relaxations for Minimizing Functions that are Summations of Small Polynomials , 2008, SIAM J. Optim..

[144]  Frank Vallentin.  Optimal distortion embeddings of distance regular graphs into Euclidean spaces , 2008, J. Comb. Theory, Ser. B.

[145]  Monique Laurent,et al.  Semidefinite Characterization and Computation of Zero-Dimensional Real Radical Ideals , 2008, Found. Comput. Math..

[146]  Raúl E. Curto,et al.  An analogue of the Riesz–Haviland theorem for the truncated moment problem , 2008 .

[147]  Ha Huy Vui,et al.  Global Optimization of Polynomials Using the Truncated Tangency Variety and Sums of Squares , 2008, SIAM J. Optim..

[148]  Markus Schweighofer.  A Gr\"obner basis proof of the flat extension theorem for moment matrices , 2008 .

[149]  M. Marshall.  Positive polynomials and sums of squares , 2008 .

[150]  Monique Laurent,et al.  Computing Semidefinite Programming Lower Bounds for the (Fractional) Chromatic Number Via Block-Diagonalization , 2008, SIAM J. Optim..

[151]  M. Laurent.  THE OPERATOR FOR THE CHROMATIC NUMBER OF AGRAPH , 2008 .

[152]  Bernard Mourrain,et al.  A generalized flat extension theorem for moment matrices , 2009 .

[153]  Jean B. Lasserre,et al.  Convexity in SemiAlgebraic Geometry and Polynomial Optimization , 2008, SIAM J. Optim..

[154]  S. Sullivant,et al.  Emerging applications of algebraic geometry , 2009 .

[155]  Jiawang Nie,et al.  Sum of squares method for sensor network localization , 2006, Comput. Optim. Appl..

[156]  Masakazu Muramatsu,et al.  A note on sparse SOS and SDP relaxations for polynomial optimization problems over symmetric cones , 2009, Comput. Optim. Appl..

[157]  Didier Henrion,et al.  GloptiPoly 3: moments, optimization and semidefinite programming , 2007, Optim. Methods Softw..

[158]  Christine Bachoc,et al.  Optimality and uniqueness of the (4, 10, 1/6) spherical code , 2007, J. Comb. Theory, Ser. A.

[159]  Christine Bachoc,et al.  Semidefinite programming, multivariate orthogonal polynomials, and codes in spherical caps , 2009, Eur. J. Comb..

[160]  M. Marshall,et al.  Representations of Non-Negative Polynomials, Degree Bounds and Applications to Optimization , 2009, Canadian Journal of Mathematics.

[161]  Didier Henrion,et al.  Semidefinite Representation of Convex Hulls of Rational Varieties , 2009, ArXiv.

[162]  Claus Scheiderer,et al.  Positivity and sums of squares: A guide to recent results , 2009 .

[163]  Konrad Schmuedgen.  Noncommutative Real Algebraic Geometry Some Basic Concepts and First Ideas , 2009 .

[164]  J. William Helton,et al.  Semidefinite representation of convex sets , 2007, Math. Program..

[165]  Alicia Dickenstein,et al.  Solving Polynomial Equations: Foundations, Algorithms, and Applications , 2010 .

[166]  Etienne de Klerk,et al.  Exploiting group symmetry in semidefinite programming relaxations of the quadratic assignment problem , 2007, Math. Program..

[167]  J. Gallier.  Quadratic Optimization Problems , 2020, Linear Algebra and Optimization with Applications to Machine Learning.