On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators

This paper shows, by means of an operator called asplitting operator, that the Douglas—Rachford splitting method for finding a zero of the sum of two monotone operators is a special case of the proximal point algorithm. Therefore, applications of Douglas—Rachford splitting, such as the alternating direction method of multipliers for convex programming decomposition, are also special cases of the proximal point algorithm. This observation allows the unification and generalization of a variety of convex programming algorithms. By introducing a modified version of the proximal point algorithm, we derive a new,generalized alternating direction method of multipliers for convex programming. Advances of this sort illustrate the power and generality gained by adopting monotone operator theory as a conceptual framework.

[1]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[2]  G. Minty On the maximal domain of a ``monotone'' function. , 1961 .

[3]  G. Minty Monotone (nonlinear) operators in Hilbert space , 1962 .

[4]  G. Minty On the monotonicity of the gradient of a convex function. , 1964 .

[5]  R. Rockafellar Characterization of the subdifferentials of convex functions , 1966 .

[6]  R. I. Kachurovskii Non-Linear Monotone Operators in Banach Spaces , 1968 .

[7]  M. Powell A method for nonlinear constraints in minimization problems , 1969 .

[8]  M. Hestenes Multiplier and gradient methods , 1969 .

[9]  R. Rockafellar Local boundedness of nonlinear, monotone operators. , 1969 .

[10]  B. Martinet,et al.  R'egularisation d''in'equations variationnelles par approximations successives , 1970 .

[11]  R. Rockafellar,et al.  On the maximal monotonicity of subdifferential mappings. , 1970 .

[12]  R. Rockafellar On the virtual convexity of the domain and range of a nonlinear maximal monotone operator , 1970 .

[13]  R. Rockafellar On the maximality of sums of nonlinear monotone operators , 1970 .

[14]  H. Brezis Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .

[15]  J. L. Webb OPERATEURS MAXIMAUX MONOTONES ET SEMI‐GROUPES DE CONTRACTIONS DANS LES ESPACES DE HILBERT , 1974 .

[16]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[17]  Dimitri P. Bertsekas,et al.  Necessary and sufficient conditions for a penalty method to be exact , 1975, Math. Program..

[18]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[19]  D. Bertsekas,et al.  Combined Primal–Dual and Penalty Methods for Convex Programming , 1976 .

[20]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[21]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[22]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[23]  H. Brezis,et al.  Produits infinis de resolvantes , 1978 .

[24]  P. Lions,et al.  Une methode iterative de resolution d’une inequation variationnelle , 1978 .

[25]  R. Rockafellar MONOTONE OPERATORS AND AUGMENTED LAGRANGIAN METHODS IN NONLINEAR PROGRAMMING , 1978 .

[26]  D. Pascali,et al.  Nonlinear mappings of monotone type , 1979 .

[27]  Gregory B. Passty Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[28]  V. Dolezal Monotone operators and applications in control and network theory , 1979 .

[29]  E. G. Gol'shtein,et al.  Modified Lagrangians in Convex Programming and their Generalizations , 1979 .

[30]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[31]  G. Marchuk Methods of Numerical Mathematics , 1982 .

[32]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[33]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[34]  J. Spingarn Partial inverse of a monotone operator , 1983 .

[35]  M. Fortin,et al.  On decomposition - coordination methods using an augmented Lagrangian , 1983 .

[36]  D. Gabay Applications of the method of multipliers to variational inequalities , 1983 .

[37]  F. Luque Asymptotic convergence analysis of the proximal point algorithm , 1984 .

[38]  Jonathan E. Spingarn,et al.  Applications of the method of partial inverses to convex programming: Decomposition , 1985, Math. Program..

[39]  J. Spingarn A primal-dual projection method for solving systems of linear inequalities , 1985 .

[40]  M. Joshi,et al.  Some topics in nonlinear functional analysis , 1985 .

[41]  C. Michelot,et al.  Sets of efficient points in a normed space , 1986 .

[42]  J. Spingarn A projection method for least-squares solutions to overdetermined systems of linear inequalities , 1987 .

[43]  R. Glowinski,et al.  Augmented Lagrangian Methods for the Solution of Variational Problems. , 1987 .

[44]  J. Lawrence,et al.  On Fixed Points of Non-Expansive Piecewise Isometric Mappings , 1987 .

[45]  Jonathan Eckstein,et al.  The Lions-Mercier splitting algorithm and the alternating direction method are instances of the proximal point algorithm , 1988 .

[46]  P. Tseng Applications of a Splitting Algorithm to Decomposition in Convex Programming , 1988 .

[47]  Roland Durier On Locally Polyhedral Convex Functions , 1988 .

[48]  C. Michelot,et al.  About the finite convergence of the proximal point algorithm , 1988 .

[49]  Jonathan Eckstein Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[50]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[51]  R. Tyrrell Rockafellar,et al.  Scenarios and Policy Aggregation in Optimization Under Uncertainty , 1991, Math. Oper. Res..

[52]  P. Tseng Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .