Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers

In this paper, we consider the monotone inclusion problem consisting of the sum of a continuous monotone map and a point-to-set maximal monotone operator with a separable two-block structure and introduce a framework of block-decomposition prox-type algorithms for solving it which allows for each one of the single-block proximal subproblems to be solved in an approximate sense. Moreover, by showing that any method in this framework is also a special instance of the hybrid proximal extragradient (HPE) method introduced by Solodov and Svaiter, we derive corresponding convergence rate results. We also describe some instances of the framework based on specific and inexpensive schemes for solving the single-block proximal subproblems. Finally, we consider some applications of our methodology to establish for the first time (i) the iteration-complexity of an algorithm for finding a zero of the sum of two arbitrary maximal monotone operators and, as a consequence, the ergodic iteration-complexity of the Douglas-...

[1]  Benar Fux Svaiter,et al.  Maximal Monotone Operators, Convex Functions and a Special Family of Enlargements , 2002 .

[2]  Samuel Burer,et al.  Optimizing a polyhedral-semidefinite relaxation of completely positive programs , 2010, Math. Program. Comput..

[3]  R. Rockafellar,et al.  On the maximal monotonicity of subdifferential mappings. , 1970 .

[4]  Benar Fux Svaiter,et al.  A family of projective splitting methods for the sum of two maximal monotone operators , 2007, Math. Program..

[5]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[6]  Samuel Burer,et al.  Solving Lift-and-Project Relaxations of Binary Integer Programs , 2006, SIAM J. Optim..

[7]  Mikhail V. Solodov,et al.  A class of decomposition methods for convex optimization and monotone variational inclusions via the hybrid inexact proximal point framework , 2004, Optim. Methods Softw..

[8]  A. Iusem,et al.  Enlargement of Monotone Operators with Applications to Variational Inequalities , 1997 .

[9]  M. Solodov,et al.  A UNIFIED FRAMEWORK FOR SOME INEXACT PROXIMAL POINT ALGORITHMS , 2001 .

[10]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[11]  Adam Ouorou.  Epsilon-proximal decomposition method , 2004, Math. Program..

[12]  O. SIAMJ.,et al.  PROX-METHOD WITH RATE OF CONVERGENCE O(1/t) FOR VARIATIONAL INEQUALITIES WITH LIPSCHITZ CONTINUOUS MONOTONE OPERATORS AND SMOOTH CONVEX-CONCAVE SADDLE POINT PROBLEMS∗ , 2004 .

[13]  Renato D. C. Monteiro,et al.  Complexity of Variants of Tseng's Modified F-B Splitting and Korpelevich's Methods for Hemivariational Inequalities with Applications to Saddle-point and Convex Optimization Problems , 2011, SIAM J. Optim..

[14]  M. Noor.  An extraresolvent method for monotone mixed variational inequalities , 1999 .

[15]  M. Solodov,et al.  A hybrid projection-proximal point algorithm. , 1998 .

[16]  M. Fukushima,et al.  "Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods" , 2010 .

[17]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[18]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[19]  Franz Rendl,et al.  A Boundary Point Method to Solve Semidefinite Programs , 2006, Computing.

[20]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[21]  Renato D. C. Monteiro,et al.  On the Complexity of the Hybrid Proximal Extragradient Method for the Iterates and the Ergodic Mean , 2010, SIAM J. Optim..

[22]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[23]  Alfredo N. Iusem,et al.  Maximal Monotone Operators , 2008 .

[24]  Franz Rendl,et al.  Regularization Methods for Semidefinite Programming , 2009, SIAM J. Optim..

[25]  Franz Rendl,et al.  An Augmented Primal-Dual Method for Linear Conic Programs , 2008, SIAM J. Optim..

[26]  Yurii Nesterov,et al.  Dual extrapolation and its applications to solving variational inequalities and related problems , 2003, Math. Program..

[27]  Benar Fux Svaiter,et al.  A Family of Enlargements of Maximal Monotone Operators , 2000 .

[28]  Benar Fux Svaiter,et al.  General Projective Splitting Methods for Sums of Maximal Monotone Operators , 2009, SIAM J. Control. Optim..

[29]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[30]  Benar Fux Svaiter,et al.  An Inexact Hybrid Generalized Proximal Point Algorithm and Some New Results on the Theory of Bregman Functions , 2000, Math. Oper. Res..

[31]  M. Solodov,et al.  A Hybrid Approximate Extragradient – Proximal Point Algorithm Using the Enlargement of a Maximal Monotone Operator , 1999 .

[32]  B. Svaiter,et al.  ε-Enlargements of Maximal Monotone Operators: Theory and Applications , 1998 .