Bregman primal–dual first-order method and application to sparse semidefinite programming

We present a new variant of the Chambolle–Pock primal–dual algorithm with Bregman distances, analyze its convergence, and apply it to the centering problem in sparse semidefinite programming. The novelty in the method is a line search procedure for selecting suitable step sizes. The line search obviates the need for estimating the norm of the constraint matrix and the strong convexity constant of the Bregman kernel. As an application, we discuss the centering problem in large-scale semidefinite programming with sparse coefficient matrices. The logarithmic barrier function for the cone of positive semidefinite completable sparse matrices is used as the distance-generating kernel. For this distance, the complexity of evaluating the Bregman proximal operator is shown to be roughly proportional to the cost of a sparse Cholesky factorization. This is much cheaper than the standard proximal operator with Euclidean distances, which requires an eigenvalue decomposition.

[1]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[2]  H. Brezis.  Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .

[3]  Charles R. Johnson,et al.  Positive definite completions of partial Hermitian matrices , 1984 .

[4]  B. Peyton,et al.  An Introduction to Chordal Graphs and Clique Trees , 1993 .

[5]  Jonathan Eckstein,et al.  Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming , 1993, Math. Oper. Res..

[6]  Marc Teboulle,et al.  Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions , 1993, SIAM J. Optim..

[7]  O. Güler,et al.  Ergodic Convergence in Proximal Point Algorithms with Bregman Functions , 1994 .

[8]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[9]  Stephen P. Boyd,et al.  A primal—dual potential reduction method for problems involving matrix inequalities , 1995, Math. Program..

[10]  Masakazu Kojima,et al.  Exploiting sparsity in primal-dual interior-point methods for semidefinite programming , 1997, Math. Program..

[11]  Brian Borchers,et al.  SDPLIB 1.1, A Library of Semidefinite Programming Test Problems , 1998 .

[12]  Xiong Zhang,et al.  Solving Large-Scale Sparse Semidefinite Programs for Combinatorial Optimization , 1999, SIAM J. Optim..

[13]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[14]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[15]  Kazuo Murota,et al.  Exploiting Sparsity in Semidefinite Programming via Matrix Completion I: General Framework , 2000, SIAM J. Optim..

[16]  Katsuki Fujisawa,et al.  Exploiting sparsity in semidefinite programming via matrix completion II: implementation and numerical results , 2003, Math. Program..

[17]  Samuel Burer,et al.  Semidefinite Programming in the Space of Partial Positive Semidefinite Matrices , 2003, SIAM J. Optim..

[18]  Stephen A. Vavasis,et al.  A Fully Sparse Implementation of a Primal-Dual Interior-Point Potential Reduction Method for Semidefinite Programming , 2004, ArXiv.

[19]  Marc Teboulle,et al.  Interior Gradient and Proximal Methods for Convex and Conic Optimization , 2006, SIAM J. Optim..

[20]  M. Kojima,et al.  Correlative Sparsity in Primal-Dual Interior-Point Methods for LP, SDP, and SOCP , 2008 .

[21]  Yinyu Ye,et al.  Algorithm 875: DSDP5—software for semidefinite programming , 2008, TOMS.

[22]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[23]  Daniel Cremers,et al.  An algorithm for minimizing the Mumford-Shah functional , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[24]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[25]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[26]  Marc Teboulle,et al.  Gradient-based algorithms with applications to signal-recovery problems , 2010, Convex Optimization in Signal Processing and Communications.

[27]  Masakazu Kojima,et al.  Exploiting sparsity in linear and nonlinear matrix inequalities via positive semidefinite matrix completion , 2011, Math. Program..

[28]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[29]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..

[30]  Martin S. Andersen,et al.  Logarithmic barriers for sparse matrix cones , 2012, Optim. Methods Softw..

[31]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[32]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[33]  Jacek Gondzio,et al.  A Matrix-Free Preconditioner for Sparse Symmetric Positive Definite Systems and Least-Squares Problems , 2013, SIAM J. Sci. Comput..

[34]  Marc Teboulle,et al.  Rate of Convergence Analysis of Decomposition Methods Based on the Proximal Method of Multipliers for Convex Minimization , 2014, SIAM J. Optim..

[35]  Yifan Sun,et al.  Decomposition in Conic Optimization with Partially Separable Structure , 2013, SIAM J. Optim..

[36]  Martin S. Andersen,et al.  Chordal Graphs and Semidefinite Optimization , 2015, Found. Trends Optim..

[37]  Javad Lavaei,et al.  ADMM for sparse semidefinite programming with applications to optimal power flow problem , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[38]  Yifan Sun,et al.  Decomposition Methods for Sparse Matrix Nearness Problems , 2015, SIAM J. Matrix Anal. Appl..

[39]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[40]  Antonin Chambolle,et al.  An introduction to continuous optimization for imaging , 2016, Acta Numerica.

[41]  Antonin Chambolle,et al.  On the ergodic convergence rates of a first-order primal–dual algorithm , 2016, Math. Program..

[42]  Yang Zheng,et al.  Fast ADMM for semidefinite programs with chordal sparsity , 2016, 2017 American Control Conference (ACC).

[43]  Anders Rantzer,et al.  Distributed Semidefinite Programming With Application to Large-Scale System Analysis , 2015, IEEE Transactions on Automatic Control.

[44]  Yurii Nesterov,et al.  Lectures on Convex Optimization , 2018 .

[45]  Ming Yan,et al.  A new primal-dual algorithm for minimizing the sum of three functions with a linear operator , 2016, 1611.09805.

[46]  Thomas Pock,et al.  A First-Order Primal-Dual Algorithm with Linesearch , 2016, SIAM J. Optim..

[47]  Timothy A. Davis,et al.  The SuiteSparse Matrix Collection Website Interface , 2019, J. Open Source Softw..

[48]  Joachim Dahl,et al.  On the robustness and scalability of semidefinite relaxation for optimal power flow problems , 2018, Optimization and Engineering.

[49]  Jacek Gondzio,et al.  An inexact dual logarithmic barrier method for solving sparse semidefinite programs , 2018, Mathematical Programming.

[50]  Lieven Vandenberghe,et al.  On the equivalence of the primal-dual hybrid gradient method and Douglas–Rachford splitting , 2018, Math. Program..

[51]  Yang Zheng,et al.  Chordal decomposition in operator-splitting methods for sparse semidefinite programs , 2017, Mathematical Programming.

[52]  Jacek Gondzio,et al.  An interior point-proximal method of multipliers for convex quadratic programming , 2019, Computational Optimization and Applications.

[53]  Jacek Gondzio,et al.  A Relaxed Interior Point Method for Low-Rank Semidefinite Programming Problems with Applications to Matrix Completion , 2019, Journal of Scientific Computing.

[54]  Antonis Papachristodoulou,et al.  Chordal and factor-width decompositions for scalable semidefinite and polynomial optimization , 2021, Annu. Rev. Control..

[55]  Shiqian Ma,et al.  An ADMM-based interior-point method for large-scale linear programming , 2018, Optim. Methods Softw..

[56]  Javad Lavaei,et al.  Sparse semidefinite programs with guaranteed near-linear time complexity via dualized clique tree conversion , 2017, Mathematical Programming.

[57]  An Interior Point-Proximal Method of Multipliers for Linear Positive Semi-Definite Programming , 2021, Journal of Optimization Theory and Applications.