O(1/t) complexity analysis of the generalized alternating direction method of multipliers

Owing to its efficiency in solving some types of large-scale separable optimization problems with linear constraints, the convergence rate of the alternating direction method of multipliers (ADMM for short) has recently attracted significant attention. In this paper, we consider the generalized ADMM (G-ADMM), which incorporates an acceleration factor and is more efficient. Instead of using a solution measure that depends on a bounded set and cannot be easily estimated, we propose using the original ϵ-optimal solution measure, under which we prove that the G-ADMM converges at a rate of O(1/t). The new bound depends on the penalty parameter and the distance between the initial point and solution set, which is more reasonable than the previous bound.

[1]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[2]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[3]  Yunmei Chen,et al.  An Accelerated Linearized Alternating Direction Method of Multipliers , 2014, SIAM J. Imaging Sci..

[4]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[5]  Guanghui Lan,et al.  An optimal method for stochastic composite optimization , 2011, Mathematical Programming.

[6]  Defeng Sun,et al.  Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Programming , 2017, Math. Oper. Res..

[7]  Daniel Boley,et al.  Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs , 2013, SIAM J. Optim..

[8]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[9]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[10]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[11]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[12]  Su Zhang,et al.  A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs , 2010, Eur. J. Oper. Res..

[13]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[14]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[15]  Damek Davis,et al.  Convergence Rate Analysis of Several Splitting Schemes , 2014, 1406.4834.

[16]  Xiaoming Yuan,et al.  Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs , 2013, SIAM J. Numer. Anal..

[17]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .

[18]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[19]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[20]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[21]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[22]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[23]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[24]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[25]  D. Bertsekas,et al.  An Alternating Direction Method for Linear Programming , 1990 .

[26]  Renato D. C. Monteiro,et al.  Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers , 2013, SIAM J. Optim..

[27]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[28]  Wei Hong Yang,et al.  Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems , 2016, SIAM J. Numer. Anal..

[29]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[30]  Tianyi Lin,et al.  On the Convergence Rate of Multi-Block ADMM , 2014, 1408.4265.