Distributed Synchronous and Asynchronous Algorithms for Semidefinite Programming With Diagonal Constraints

This paper develops distributed synchronous and asynchronous algorithms for the large-scale semi-definite programming with diagonal constraints, which has wide applications in combination optimization, image processing and community detection. The information of the semi-definite programming is allocated to multiple interconnected agents such that each agent aims to find a solution by communicating to its neighbors. Based on low-rank property of solutions and the Burer-Monteiro factorization, we transform the original problem into a distributed optimization problem over unit spheres to reduce variable dimensions and ensure positive semi-definiteness without involving semi-definite projections, which are computationally expensive. For the distributed optimization problem, we propose distributed synchronous and asynchronous algorithms, both of which reduce computational burden and storage space compared with existing centralized algorithms. Specifically, the distributed synchronous algorithm almost surely escapes strict saddle points and converges to the set of optimal solutions to the optimization problem. In addition, the proposed distributed asynchronous algorithm allows communication delays and converges to the set of critical points to the optimization problem under mild conditions. By applying proposed algorithms to image segmentation applications, we illustrate the efficiency and convergence performance of the two proposed algorithms.

[1]  Francis R. Bach,et al.  Low-Rank Optimization on the Cone of Positive Semidefinite Matrices , 2008, SIAM J. Optim..

[2]  Wenwu Yu,et al.  Distributed fixed step-size algorithm for dynamic economic dispatch with power flow limits , 2020, Science China Information Sciences.

[3]  Imad M. Jaimoukha,et al.  A Semidefinite Relaxation Procedure for Fault-Tolerant Observer Design , 2015, IEEE Transactions on Automatic Control.

[4]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[5]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Emmanuel Abbe,et al.  Exact Recovery in the Stochastic Block Model , 2014, IEEE Transactions on Information Theory.

[7]  Nicolas Boumal,et al.  The non-convex Burer-Monteiro approach works on smooth semidefinite programs , 2016, NIPS.

[8]  Yll Haxhimusa,et al.  Estimation of Distribution Algorithm for the Max-Cut Problem , 2013, GbRPR.

[9]  M. Shub Global Stability of Dynamical Systems , 1986 .

[10]  Haibo Ji,et al.  Distributed Design for Nuclear Norm Minimization of Linear Matrix Equations With Constraints , 2020, IEEE Transactions on Automatic Control.

[11]  Yiguang Hong,et al.  Multi-Agent Optimization Design for Autonomous Lagrangian Systems , 2016, Unmanned Syst..

[12]  Daoqiang Zhang,et al.  Semisupervised Kernel Matrix Learning by Kernel Propagation , 2010, IEEE Transactions on Neural Networks.

[13]  Georgios B. Giannakis,et al.  Distributed Optimal Power Flow for Smart Microgrids , 2012, IEEE Transactions on Smart Grid.

[14]  Shu Liang,et al.  Distributed Smooth Convex Optimization With Coupled Constraints , 2020, IEEE Transactions on Automatic Control.

[15]  Xiong Zhang,et al.  Solving Large-Scale Sparse Semidefinite Programs for Combinatorial Optimization , 1999, SIAM J. Optim..

[16]  Gábor Pataki,et al.  On the Rank of Extreme Matrices in Semidefinite Programs and the Multiplicity of Optimal Eigenvalues , 1998, Math. Oper. Res..

[17]  Anders Rantzer,et al.  Distributed Semidefinite Programming With Application to Large-Scale System Analysis , 2015, IEEE Transactions on Automatic Control.

[18]  Brian D. O. Anderson,et al.  Network Flows That Solve Linear Equations , 2015, IEEE Transactions on Automatic Control.

[19]  Mingwang Zhang,et al.  A second order Mehrotra-type predictor-corrector algorithm for semidefinite optimization , 2012, Journal of Systems Science and Complexity.

[20]  Isak Nielsen,et al.  Distributed primal–dual interior-point methods for solving tree-structured coupled convex problems using message-passing , 2017, Optim. Methods Softw..

[21]  Brian Borchers,et al.  Implementation of a primal–dual method for SDP on a shared memory parallel architecture , 2007, Comput. Optim. Appl..

[22]  Georgios B. Giannakis,et al.  Power System Nonlinear State Estimation Using Distributed Semidefinite Programming , 2014, IEEE Journal of Selected Topics in Signal Processing.

[23]  Javad Lavaei,et al.  A fast distributed algorithm for decomposable semidefinite programs , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[24]  Lei Wang,et al.  Scalable Large-Margin Mahalanobis Distance Metric Learning , 2010, IEEE Transactions on Neural Networks.

[25]  BoykovYuri,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006 .

[26]  Xiaoming Hu,et al.  Interval Consensus for Multiagent Networks , 2020, IEEE Transactions on Automatic Control.

[27]  Thomas Kappeler,et al.  On the Regularity of the Composition of Diffeomorphisms , 2011, 1107.0488.

[28]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yiguang Hong,et al.  Distributed Computation for Solving the Sylvester Equation Based on Optimization , 2019, IEEE Control Systems Letters.

[30]  Jaeseok Kim,et al.  Exact ML Criterion Based on Semidefinite Relaxation for MIMO Systems , 2014, IEEE Signal Processing Letters.

[31]  Javad Lavaei,et al.  ADMM for sparse semidefinite programming with applications to optimal power flow problem , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[32]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[33]  Michael I. Jordan,et al.  First-order methods almost always avoid strict saddle points , 2019, Mathematical Programming.

[34]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[35]  Jie Chen,et al.  Distributed Solver for Discrete-Time Lyapunov Equations Over Dynamic Networks With Linear Convergence Rate , 2019, IEEE Transactions on Cybernetics.

[36]  Faliu Yi,et al.  Image segmentation: A survey of graph-cut methods , 2012, 2012 International Conference on Systems and Informatics (ICSAI2012).

[37]  Nicolas Boumal,et al.  On the low-rank approach for semidefinite programs arising in synchronization and community detection , 2016, COLT.

[38]  Sonia Martínez,et al.  Scheduled-Asynchronous Distributed Algorithm for Optimal Power Flow , 2017, IEEE Transactions on Control of Network Systems.

[39]  Yin Tat Lee,et al.  A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[40]  R. A. Jabr,et al.  Solution to Economic Dispatching With Disjoint Feasible Regions Via Semidefinite Programming , 2012, IEEE Transactions on Power Systems.

[41]  Asuman E. Ozdaglar,et al.  Convergence rate of block-coordinate maximization Burer–Monteiro method for solving large SDPs , 2018, Mathematical Programming.

[42]  Antonis Papachristodoulou,et al.  Chordal sparsity, decomposing SDPs and the Lyapunov equation , 2014, 2014 American Control Conference.

[43]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Brian D. O. Anderson,et al.  Network Flows That Solve Sylvester Matrix Equations , 2019, IEEE Transactions on Automatic Control.

[45]  Vladimir Kolmogorov,et al.  Graph cut based image segmentation with connectivity priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Michael W. Mahoney,et al.  Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments , 2015, Proceedings of the IEEE.

[47]  En-Liang Hu,et al.  Low-Rank Matrix Learning Using Biconvex Surrogate Minimization , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[48]  Andrea Montanari,et al.  Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality , 2017, COLT.

[49]  J. Z. Kolter,et al.  The Mixing method: low-rank coordinate descent for semidefinite programming with diagonal constraints , 2017 .

[50]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[51]  Pablo A. Parrilo,et al.  Semidefinite Programming Relaxations and Algebraic Optimization in Control , 2003, Eur. J. Control.

[52]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..

[53]  Yiguang Hong,et al.  Distributed Optimization Design for Computation of Algebraic Riccati Inequalities , 2020, IEEE Transactions on Cybernetics.