Exploiting ideal-sparsity in the generalized moment problem with application to matrix factorization ranks

We explore a new type of sparsity for the generalized moment problem (GMP) that we call ideal-sparsity . This sparsity exploits the presence of equality constraints requiring the measure to be supported on the variety of an ideal generated by bilinear monomials modeled by an associated graph. We show that this enables an equivalent sparse reformulation of the GMP, where the single (high dimensional) measure variable is replaced by several (lower dimensional) measure variables supported on the maximal cliques of the graph. We explore the resulting hierarchies of moment-based relaxations for the original dense formulation of GMP and this new, equivalent ideal-sparse reformulation, when applied to the problem of bounding nonnegative- and completely positive matrix factorization ranks. We show that the ideal-sparse hierarchies provide bounds that are at least as good (and often tighter) as those obtained from the dense hierarchy. This is in sharp contrast to the situation when exploiting correlative sparsity, as is most common in the literature, where the resulting bounds are weaker than the dense bounds. Moreover, while correlative sparsity requires the underlying graph to be chordal, no such assumption is needed for ideal-sparsity. Numerical results show that the ideal-sparse bounds are often tighter and much faster to compute than their dense analogs.

[1]  Jie Wang,et al.  Sparse Polynomial Optimization: Theory and Practice , 2022, ArXiv.

[2]  Monique Laurent,et al.  Bounding the separable rank via polynomial optimization , 2021, Linear Algebra and its Applications.

[3]  Milan Korda,et al.  Stability and Performance Verification of Dynamical Systems Controlled by Neural Networks: Algorithms and Complexity , 2021, IEEE Control Systems Letters.

[4]  J. Lasserre,et al.  CS-TSSOS: Correlative and Term Sparsity for Large-Scale Polynomial Optimization , 2020, ACM Trans. Math. Softw..

[5]  Antonis Papachristodoulou,et al.  Chordal and factor-width decompositions for scalable semidefinite and polynomial optimization , 2021, Annu. Rev. Control..

[6]  Emiliano Traversi,et al.  A Semidefinite Optimization-based Branch-and-Bound Algorithm for Several Reactive Optimal Power Flow Problems , 2021, ArXiv.

[7]  E. Andersen,et al.  A Clique Merging Algorithm to Solve Semidefinite Relaxations of Optimal Power Flow Problems , 2021, IEEE Transactions on Power Systems.

[8]  Giovanni Fantuzzi,et al.  Sum-of-squares chordal decomposition of polynomial matrix inequalities , 2020, Mathematical Programming.

[9]  Victor Magron,et al.  Chordal-TSSOS: A Moment-SOS Hierarchy That Exploits Term Sparsity with Chordal Extension , 2020, SIAM J. Optim..

[10]  J. Lasserre,et al.  TSSOS: A Moment-SOS Hierarchy That Exploits Term Sparsity , 2019, SIAM Journal on Optimization.

[11]  J. Povh,et al.  Sparse noncommutative polynomial optimization , 2019, Mathematical Programming.

[12]  A. Ivic Sums of squares , 2020, An Introduction to 𝑞-analysis.

[13]  Victor Magron,et al.  Semialgebraic Optimization for Lipschitz Constants of ReLU Networks , 2020, NeurIPS.

[14]  Mark Cannon,et al.  A clique graph based merging strategy for decomposable SDPs. , 2019 .

[15]  Jie Wang,et al.  A second order cone characterization for sums of nonnegative circuits , 2019, ISSAC.

[16]  Jean Lasserre,et al.  The Moment-SOS Hierarchy , 2018, Series on Optimization and Its Applications.

[17]  S. Pearson Moments , 2020, Narrative inquiry in bioethics.

[18]  Jean B. Lasserre,et al.  Semidefinite Relaxations for Lebesgue and Gaussian Measures of Unions of Basic Semialgebraic Sets , 2019, Math. Oper. Res..

[19]  E. D. Klerk,et al.  A Survey of Semidefinite Programming Approaches to the Generalized Problem of Moments and Their Error Analysis , 2018, Association for Women in Mathematics Series.

[20]  Gennadiy Averkov,et al.  Optimal Size of Linear Matrix Inequalities in Semidefinite Approaches to Polynomial Optimization , 2018, SIAM J. Appl. Algebra Geom..

[21]  Monique Laurent,et al.  Lower Bounds on Matrix Factorization Ranks via Noncommutative Polynomial Optimization , 2017, Found. Comput. Math..

[22]  Daniel K. Molzahn,et al.  Lasserre Hierarchy for Large Scale Polynomial Optimization in Real and Complex Variables , 2017, SIAM J. Optim..

[23]  D. Henrion,et al.  Semidefinite Approximations of Invariant Measures for Polynomial Systems , 2018, 1807.00754.

[24]  Jiawang Nie,et al.  Symmetric Tensor Nuclear Norms , 2016, SIAM J. Appl. Algebra Geom..

[25]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..

[26]  Colin Neil Jones,et al.  Stability and performance verification of optimization-based controllers , 2015, Autom..

[27]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[28]  Frank Vallentin,et al.  On the Turing Model Complexity of Interior Point Methods for Semidefinite Programming , 2015, SIAM J. Optim..

[29]  Parikshit Shah,et al.  Relative Entropy Relaxations for Signomial Optimization , 2014, SIAM J. Optim..

[30]  Pablo A. Parrilo,et al.  Self-scaled bounds for atomic cone ranks: applications to nonnegative rank and cp-rank , 2014, Math. Program..

[31]  Parikshit Shah,et al.  Guaranteed Tensor Decomposition: A Moment Approach , 2015, ICML.

[32]  J. Lasserre An Introduction to Polynomial and Semi-Algebraic Optimization , 2015 .

[33]  Immanuel M. Bomze,et al.  New Lower Bounds and Asymptotics for the cp-Rank , 2015, SIAM J. Matrix Anal. Appl..

[34]  Immanuel M. Bomze,et al.  From seven to eleven: Completely positive matrices with high cp-rank , 2014 .

[35]  Peter J. C. Dickinson,et al.  On the computational complexity of membership problems for the completely positive cone and its dual , 2014, Comput. Optim. Appl..

[36]  Timo de Wolff,et al.  Amoebas, nonnegative polynomials and sums of squares supported on circuits , 2014, 1402.0462.

[37]  Jesse T. Holzer,et al.  Implementation of a Large-Scale Optimal Power Flow Solver Based on Semidefinite Programming , 2013, IEEE Transactions on Power Systems.

[38]  Florian Jarre,et al.  On the cp-Rank and Minimal cp Factorizations of a Completely Positive Matrix , 2013, SIAM J. Matrix Anal. Appl..

[39]  Didier Henrion,et al.  Measures and LMIs for optimal control of piecewise-affine systems , 2012, 2013 European Control Conference (ECC).

[40]  Thorsten Theobald,et al.  Exploiting Symmetries in SDP-Relaxations for Polynomial Optimization , 2011, Math. Oper. Res..

[41]  Jiawang Nie,et al.  The A-Truncated K -Moment Problem , 2012 .

[42]  Samuel Fiorini,et al.  Approximation Limits of Linear Programs (Beyond Hierarchies) , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[43]  Nicolas Gillis,et al.  On the Geometric Interpretation of the Nonnegative Rank , 2010, 1009.0880.

[44]  Arie M. C. A. Koster,et al.  Treewidth computations I. Upper bounds , 2010, Inf. Comput..

[45]  LeRoy B. Beasley,et al.  Real rank versus nonnegative rank , 2009 .

[46]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[47]  R. Cottle Linear Complementarity Problem , 2009, Encyclopedia of Optimization.

[48]  M. Laurent Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[49]  Jean B. Lasserre,et al.  A semidefinite programming approach to the generalized problem of moments , 2007, Math. Program..

[50]  Jean B. Lasserre,et al.  Convergent SDP-Relaxations for Polynomial Optimization with Sparsity , 2006, ICMS.

[51]  Masakazu Muramatsu,et al.  Sums of Squares and Semidefinite Programming Relaxations for Polynomial Optimization Problems with Structured Sparsity , 2004 .

[52]  Abraham Berman,et al.  The maximal cp-rank of rank k completely positive matrices , 2003 .

[53]  J. Lasserre,et al.  Detecting global optimality and extracting solutions in GloptiPoly , 2003 .

[54]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[55]  R. Curto,et al.  The truncated complex -moment problem , 2000 .

[56]  Knud D. Andersen,et al.  The Mosek Interior Point Optimizer for Linear Programming: An Implementation of the Homogeneous Algorithm , 2000 .

[57]  Francesco Barioli Completely positive matrices with a book-graph , 1998 .

[58]  Shu-huang Xiang,et al.  Notes on completely positive matrices , 1998 .

[59]  Raúl E. Curto,et al.  Solution of the Truncated Complex Moment Problem for Flat Data , 1996 .

[60]  Charles R. Johnson,et al.  Completely positive matrices associated with M-matrices , 1994 .

[61]  Luigi Salce,et al.  Completely positive matrices and positivity of least squares solutions , 1993 .

[62]  L. Rodman,et al.  Positive semidefinite matrices with a given sparsity pattern , 1988 .

[63]  Derek G. Corneil,et al.  Complexity of finding embeddings in a k -tree , 1987 .

[64]  B. Reznick Extremal PSD forms with few terms , 1978 .

[65]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[66]  C. Bron,et al.  Algorithm 457: finding all cliques of an undirected graph , 1973 .

[67]  Frank Harary,et al.  Graph Theory , 2016 .

[68]  P. Erdös,et al.  The Representation of a Graph by Set Intersections , 1966, Canadian Journal of Mathematics.