On the O(1/t) Convergence Rate of Alternating Direction Method with Logarithmic-Quadratic Proximal Regularization

It was shown recently that the Douglas--Rachford alternating direction method of multipliers can be combined with the logarithmic-quadratic proximal regularization for solving a class of variational inequalities with separable structures. This paper further shows a worst-case $O(1/t)$ convergence rate for this algorithm where a general Glowinski relaxation factor is used.

[1]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[2]  Xiaoming Yuan.  The prediction-correction approach to nonlinear complementarity problems , 2007, Eur. J. Oper. Res..

[3]  Michael K. Ng,et al.  Inexact Alternating Direction Methods for Image Recovery , 2011, SIAM J. Sci. Comput..

[4]  Min Li,et al.  An LQP-Based Decomposition Method for Solving a Class of Variational Inequalities , 2011, SIAM J. Optim..

[5]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[6]  R. Glowinski,et al.  Finite element approximation and iterative solution of a class of mildly non-linear elliptic equations , 1978 .

[7]  W. Hager,et al.  Large Scale Optimization : State of the Art , 1993 .

[8]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[9]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[10]  Marc Teboulle,et al.  A Logarithmic-Quadratic Proximal Method for Variational Inequalities , 1999, Comput. Optim. Appl..

[11]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[12]  Simon Setzer,et al.  Split Bregman Algorithm, Douglas-Rachford Splitting and Frame Shrinkage , 2009, SSVM.

[13]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[14]  Xiaoming Yuan,et al.  Alternating Direction Method for Covariance Selection Models , 2011, Journal of Scientific Computing.

[15]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[16]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[17]  Raymond H. Chan,et al.  Alternating Direction Method for Image Inpainting in Wavelet Domains , 2011, SIAM J. Imaging Sci..

[18]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[19]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[20]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[21]  Bingsheng He,et al.  On the O(1/t) convergence rate of the projection and contraction methods for variational inequalities with Lipschitz continuous monotone operators , 2013, Computational Optimization and Applications.

[22]  Renato D. C. Monteiro,et al.  Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers , 2013, SIAM J. Optim..

[23]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[24]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[25]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[26]  O. SIAMJ.,et al.  PROX-METHOD WITH RATE OF CONVERGENCE O(1/t) FOR VARIATIONAL INEQUALITIES WITH LIPSCHITZ CONTINUOUS MONOTONE OPERATORS AND SMOOTH CONVEX-CONCAVE SADDLE POINT PROBLEMS∗ , 2004 .

[27]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[28]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[29]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[30]  Su Zhang,et al.  A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs , 2010, Eur. J. Oper. Res..

[31]  Marc Teboulle,et al.  Lagrangian Duality and Related Multiplier Methods for Variational Inequality Problems , 1999, SIAM J. Optim..

[32]  Xiaoming Yuan,et al.  Alternating Direction Methods for Sparse Covariance Selection * , 2009 .

[33]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[34]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[35]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[36]  Bingsheng He,et al.  A Logarithmic-Quadratic Proximal Prediction-Correction Method for Structured Monotone Variational Inequalities , 2006, Comput. Optim. Appl..

[37]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[38]  Peter Richtárik,et al.  Smooth minimization of nonsmooth functions with parallel coordinate descent methods , 2013, Modeling and Optimization: Theory and Applications.

[39]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .