On the convergence properties of a majorized ADMM for linearly constrained convex optimization problems with coupled objective functions

In this paper, we establish the convergence properties for a majorized alternating direction method of multipliers (ADMM) for linearly constrained convex optimization problems whose objectives contain coupled functions. Our convergence analysis relies on the generalized MeanValue Theorem which plays an important role to properly control the cross terms due to the presence of coupled objective functions. Our results in particular show that directly applying 2-block ADMM with a large step length to the linearly constrained convex optimization problem with a quadratically coupled objective function is convergent under mild conditions. We also provide several iteration complexity results for the algorithm.

[1]  R. Glowinski.  Lectures on Numerical Methods for Non-Linear Variational Problems , 1981 .

[2]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[3]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[4]  Kim-Chuan Toh,et al.  A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions , 2014, Mathematical Programming.

[5]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[6]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[7]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[8]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[9]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[10]  Damek Davis,et al.  Convergence Rate Analysis of Several Splitting Schemes , 2014, 1406.4834.

[11]  Kim-Chuan Toh,et al.  A Majorized ADMM with Indefinite Proximal Terms for Linearly Constrained Convex Composite Optimization , 2014, SIAM J. Optim..

[12]  J. Hiriart-Urruty,et al.  Generalized Hessian matrix and second-order optimality conditions for problems withC1,1 data , 1984 .

[13]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[14]  Renato D. C. Monteiro,et al.  Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers , 2013, SIAM J. Optim..

[15]  Jonathan Eckstein,et al.  Understanding the Convergence of the Alternating Direction Method of Multipliers: Theoretical and Computational Perspectives , 2015 .

[16]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .