A Linearized Alternating Direction Method of Multipliers with Substitution Procedure

We consider the linearly constrained separable convex programming problem whose objective function is separable into m individual convex functions with non-overlapping variables. The alternating direction method of multipliers (ADMM) has been well studied in the literature for the special case m = 2, but the direct extension of ADMM for the general case m ≥ 2 is not necessarily convergent. In this paper, we propose a new linearized ADMM-based contraction type algorithms for the general case m ≥ 2. For the proposed algorithm, we prove its convergence via the analytic framework of contractive type methods and we derive a worst-case O(1/t) convergence rate in ergodic sense. Finally, numerical results are reported to demonstrate the effectiveness of the proposed algorithm.

[1]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[2]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[3]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[4]  Xiaoming Yuan,et al.  Convergence rate and iteration complexity on the alternating direction method of multipliers with a substitution procedure for separable convex programming , 2012 .

[5]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[6]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[7]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[8]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[9]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[10]  W. Oettli,et al.  Mathematische Optimierung : Grundlagen und Verfahren , 1975 .

[11]  Jonathan Eckstein.  Some Saddle-function splitting methods for convex programming , 1994 .

[12]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[13]  Xiaoming Yuan,et al.  A Note on the Alternating Direction Method of Multipliers , 2012, J. Optim. Theory Appl..

[14]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[15]  R. Rockafellar.  Convex Analysis: (pms-28) , 1970 .

[16]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[17]  Francisco Facchinei,et al.  Decomposition algorithms for generalized potential games , 2011, Comput. Optim. Appl..

[18]  Xingju Cai,et al.  Convergence Analysis of Alternating Direction Method of Multipliers for a Class of Separable Convex Programming , 2013 .