On the Step Size of Symmetric Alternating Directions Method of Multipliers

The alternating direction method of multipliers (ADMM) is an application of the Douglas-Rachford splitting method; and the symmetric version of ADMM which updates the Lagrange multiplier twice at each iteration is an application of the Peaceman-Rachford splitting method. Sometimes the symmetric ADMM works empirically; but theoretically its convergence is not guaranteed. It was recently found that the convergence of symmetric ADMM can be sufficiently ensured if both the step sizes for updating the Lagrange multiplier are shrank conservatively. In this paper, we focus on the convex programming context and specify a step size domain that can ensure the convergence of symmetric ADMM. In particular, it is shown for the first time that, we can choose Glowinski’s larger step size for one of the Lagrange-multiplier-updating steps at each iteration; so shrinking both the step sizes is not necessary for the symmetric ADMM. Some known results in the ADMM literature turn out to be special cases of our discussion.

[1]  A. Latif,et al.  On the O(1/t)$O(1/t)$ convergence rate of the alternating direction method with LQP regularization for solving structured variational inequality problems , 2016 .

[2]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[3]  Bingsheng He,et al.  Generalized alternating direction method of multipliers: new theoretical insights and applications , 2015, Math. Program. Comput..

[4]  Xiaoming Yuan,et al.  A Generalized Proximal Point Algorithm and Its Convergence Rate , 2014, SIAM J. Optim..

[5]  Bingsheng He,et al.  A Strictly Contractive Peaceman-Rachford Splitting Method for Convex Programming , 2014, SIAM J. Optim..

[6]  Roland Glowinski,et al.  On Alternating Direction Methods of Multipliers: A Historical Perspective , 2014, Modeling, Simulation and Optimization for Science and Technology.

[7]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[8]  Jonathan Eckstein.  Augmented Lagrangian and Alternating Direction Methods for Convex Optimization: A Tutorial and Some Illustrative Computational Results , 2012 .

[9]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[10]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[11]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[12]  Masao Fukushima,et al.  Some Reformulations and Applications of the Alternating Direction Method of Multipliers , 1994 .

[13]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[14]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[15]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[16]  R. Glowinski.  Lectures on Numerical Methods for Non-Linear Variational Problems , 1981 .

[17]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[18]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[19]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[20]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .

[21]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[22]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .