Scalable preconditioning of block-structured linear algebra systems using ADMM

We study the solution of block-structured linear algebra systems arising in optimization by using iterative solution techniques. These systems are the core computational bottleneck of many problems of interest such as parameter estimation, optimal control, network optimization, and stochastic programming. Our approach uses a Krylov solver (GMRES) that is preconditioned with an alternating method of multipliers (ADMM). We show that this ADMM-GMRES approach overcomes well-known scalability issues of Schur complement decomposition in problems that exhibit a high degree of coupling. The effectiveness of the approach is demonstrated using linear systems that arise in stochastic optimal power flow problems and that contain up to 2 million total variables and 4,000 coupling variables. We find that ADMM-GMRES is nearly an order of magnitude faster than Schur complement decomposition. Moreover, we demonstrate that the approach is robust to the selection of the augmented Lagrangian penalty parameter, which is a key advantage over the direct use of ADMM.

[1]  Jacob K. White,et al.  Parameter Insensitivity in ADMM-Preconditioned Solution of Saddle-Point Problems , 2016, 1602.02135.

[2]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[3]  Deren Han,et al.  Convergence of ADMM for multi-block nonconvex separable optimization models , 2017 .

[4]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[5]  Victor M. Zavala,et al.  Structured nonconvex optimization of large-scale energy systems using PIPS-NLP , 2014, 2014 Power Systems Computation Conference.

[6]  Gene H. Golub,et al.  On Solving Block-Structured Indefinite Linear Systems , 2003, SIAM J. Sci. Comput..

[7]  Xiaoming Yuan,et al.  Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs , 2013, SIAM J. Numer. Anal..

[8]  Fuzhen Zhang.  The Schur complement and its applications , 2005 .

[9]  W. Marsden.  I and J , 2012 .

[10]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[11]  Patrick Amestoy,et al.  A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling , 2001, SIAM J. Matrix Anal. Appl..

[12]  Victor M. Zavala,et al.  An Augmented Lagrangian Filter Method for Real-Time Embedded Optimization , 2017, IEEE Transactions on Automatic Control.

[13]  Qingqing Zheng,et al.  The corrected Uzawa method for solving saddle point problems , 2015, Numer. Linear Algebra Appl..

[14]  David L. Woodruff,et al.  Pyomo — Optimization Modeling in Python , 2012, Springer Optimization and Its Applications.

[15]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[16]  Michael A. Saunders,et al.  Preconditioners for Indefinite Systems Arising in Optimization , 1992, SIAM J. Matrix Anal. Appl..

[17]  Jacek Gondzio,et al.  Parallel interior-point solver for structured linear programs , 2003, Math. Program..

[18]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[19]  G. Golub,et al.  Inexact and preconditioned Uzawa algorithms for saddle point problems , 1994 .

[20]  Vincent Pateloup,et al.  Study of a primal-dual algorithm for equality constrained minimization , 2011, Comput. Optim. Appl..

[21]  Anders Forsgren,et al.  Iterative Solution of Augmented Systems Arising in Interior Methods , 2007, SIAM J. Optim..

[22]  Christof Büskens,et al.  A primal–dual augmented Lagrangian penalty-interior-point filter line search algorithm , 2018, Math. Methods Oper. Res..

[23]  Victor M. Zavala,et al.  Interior-point decomposition approaches for parallel solution of large-scale nonlinear parameter estimation problems , 2008 .

[24]  Brendt Wohlberg,et al.  ADMM Penalty Parameter Selection by Residual Balancing , 2017, ArXiv.

[25]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[26]  Carl D. Laird,et al.  Efficient parallel solution of large-scale nonlinear dynamic optimization problems , 2014, Comput. Optim. Appl..

[27]  Jacob K. White,et al.  GMRES-Accelerated ADMM for Quadratic Objectives , 2016, SIAM J. Optim..

[28]  Ragnar Winther,et al.  A Preconditioned Iterative Method for Saddlepoint Problems , 1992, SIAM J. Matrix Anal. Appl..

[29]  R D Zimmerman,et al.  MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education , 2011, IEEE Transactions on Power Systems.

[30]  Carl D. Laird,et al.  An interior-point method for efficient solution of block-structured NLP problems using an implicit Schur-complement decomposition , 2014, Comput. Chem. Eng..

[31]  Victor M. Zavala,et al.  Clustering-based preconditioning for stochastic programs , 2016, Comput. Optim. Appl..

[32]  Michele Benzi,et al.  On the eigenvalues of a class of saddle point matrices , 2006, Numerische Mathematik.

[33]  Euhanna Ghadimi,et al.  Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems , 2013, IEEE Transactions on Automatic Control.

[34]  Jorge Nocedal,et al.  Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..

[35]  Jacek Gondzio,et al.  Exploiting structure in parallel implementation of interior point methods for optimization , 2009, Comput. Manag. Sci..

[36]  Patrick Amestoy,et al.  Hybrid scheduling for the parallel solution of linear systems , 2006, Parallel Comput..

[37]  Shiqian Ma,et al.  Fast Multiple-Splitting Algorithms for Convex Optimization , 2009, SIAM J. Optim..

[38]  Victor M. Zavala,et al.  Benchmarking ADMM in nonconvex NLPs , 2018, Comput. Chem. Eng..

[39]  Nico Schlomer,et al.  Preconditioned Recycling Krylov subspace methods for self-adjoint problems , 2012 .

[40]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[41]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[42]  Riadh Omheni,et al.  A globally and quadratically convergent primal–dual augmented Lagrangian algorithm for equality constrained optimization , 2017, Optim. Methods Softw..

[43]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[44]  Walter Zulehner,et al.  Analysis of iterative methods for saddle point problems: a unified approach , 2002, Math. Comput..