A Globally Convergent Distributed Jacobi Scheme for Block-Structured Nonconvex Constrained Optimization Problems

Motivated by the increasing availability of highperformance parallel computing, we design a distributed parallel algorithm for linearly-coupled block-structured nonconvex constrained optimization problems. Our algorithm performs Jacobitype proximal updates of the augmented Lagrangian function, requiring only local solutions of separable block nonlinear programming (NLP) problems. We provide a cheap and explicitly computable Lyapunov function that allows us to establish global and local sublinear convergence of our algorithm, its iteration complexity, as well as simple, practical and theoretically convergent rules for automatically tuning its parameters. This in contrast to existing algorithms for nonconvex constrained optimization based on the alternating direction method of multipliers that rely on at least one of the following: Gauss-Seidel or sequential updates, global solutions of NLP problems, noncomputable Lyapunov functions, and hand-tuning of parameters. Numerical experiments showcase its advantages for large-scale problems, including the multi-period optimization of a 9000-bus AC optimal power flow test case over 168 time periods, solved on the Summit supercomputer using an open-source Julia code.

[2]  Moritz Diehl,et al.  An Augmented Lagrangian Based Algorithm for Distributed NonConvex Optimization , 2016, SIAM J. Optim..

[3]  Michael M. Zavlanos,et al.  On the Convergence of a Distributed Augmented Lagrangian Method for Nonconvex Optimization , 2017, IEEE Transactions on Automatic Control.

[4]  Lei Zhao,et al.  A First-Order Primal-Dual Method for Nonconvex Constrained Optimization Based on the Augmented Lagrangian , 2020, Mathematics of Operations Research.

[5]  Ian A. Hiskens,et al.  A Survey of Relaxations and Approximations of the Power Flow Equations , 2019, Foundations and Trends® in Electric Energy Systems.

[6]  Bronis R. de Supinski,et al.  The Design, Deployment, and Evaluation of the CORAL Pre-Exascale Systems , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.

[7]  Cesar A. Silva-Monroy,et al.  The Unit Commitment Problem With AC Optimal Power Flow Constraints , 2016, IEEE Transactions on Power Systems.

[8]  Olaf Schenk,et al.  Toward the Next Generation of Multiperiod Optimal Power Flow Solvers , 2018, IEEE Transactions on Power Systems.

[9]  Kostas Margellos,et al.  On the Convergence of a Regularized Jacobi Algorithm for Convex Optimization , 2018, IEEE Transactions on Automatic Control.

[10]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[11]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[12]  Luigi Glielmo,et al.  Optimal Power Flow model with energy storage, an extension towards large integration of renewable energy sources. , 2014 .

[13]  Guoqiang Hu,et al.  A Proximal Linearization-based Decentralized Method for Nonconvex Problems with Nonlinear Constraints , 2020, ArXiv.

[14]  Wotao Yin,et al.  Parallel Multi-Block ADMM with o(1 / k) Convergence , 2013, Journal of Scientific Computing.

[15]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[16]  Shiqian Ma,et al.  On the Global Linear Convergence of the ADMM with MultiBlock Variables , 2014, SIAM J. Optim..

[17]  Anthony V. Fiacco,et al.  Sensitivity analysis for nonlinear programming using penalty methods , 1976, Math. Program..

[18]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[19]  Colin Neil Jones,et al.  A Parametric Nonconvex Decomposition Algorithm for Real-Time and Distributed NMPC , 2016, IEEE Transactions on Automatic Control.

[20]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[21]  Prodromos Daoutidis,et al.  Fast and stable nonconvex constrained distributed optimization: the ELLADA algorithm , 2020, Optimization and Engineering.

[22]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[23]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[24]  R D Zimmerman,et al.  MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education , 2011, IEEE Transactions on Power Systems.

[25]  Jefferson G. Melo,et al.  Iteration-complexity of a Jacobi-type non-Euclidean ADMM for multi-block linearly constrained nonconvex programs , 2017, 1705.07229.

[26]  Qinghua Liu,et al.  Linearized ADMM for Nonconvex Nonsmooth Optimization With Convergence Analysis , 2017, IEEE Access.

[27]  Steffen Rebennack,et al.  An introduction to optimal power flow: Theory, formulation, and examples , 2016 .

[28]  X. Andy Sun,et al.  A two-level distributed algorithm for nonconvex constrained optimization , 2019, Computational Optimization and Applications.

[29]  Shiqian Ma,et al.  Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis , 2016, Computational Optimization and Applications.

[30]  G. Martin,et al.  Nonlinear model predictive control , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[31]  Xinlei Yi,et al.  Linear Convergence of First- and Zeroth-Order Primal–Dual Algorithms for Distributed Nonconvex Optimization , 2019, IEEE Transactions on Automatic Control.

[32]  Francisco Facchinei,et al.  Parallel and Distributed Methods for Constrained Nonconvex Optimization—Part I: Theory , 2016, IEEE Transactions on Signal Processing.

[33]  WächterAndreas,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006 .

[34]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[35]  Carlo Fischione,et al.  On the Convergence of Alternating Direction Lagrangian Methods for Nonconvex Structured Optimization Problems , 2014, IEEE Transactions on Control of Network Systems.

[36]  Jacek Gondzio,et al.  Exploiting structure in parallel implementation of interior point methods for optimization , 2009, Comput. Manag. Sci..

[37]  Victor M. Zavala,et al.  Structured nonconvex optimization of large-scale energy systems using PIPS-NLP , 2014, 2014 Power Systems Computation Conference.

[38]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Wotao Yin,et al.  Global Convergence of ADMM in Nonconvex Nonsmooth Optimization , 2015, Journal of Scientific Computing.

[40]  Mingyi Hong,et al.  Penalty Dual Decomposition Method for Nonsmooth Nonconvex Optimization—Part I: Algorithms and Convergence Analysis , 2020, IEEE Transactions on Signal Processing.

[41]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..