Iterative construction of the resolvent of a sum of maximal monotone operators

We propose two inexact parallel splitting algorithms for computing the resolvent of a weighted sum of maximal monotone operators in a Hilbert space and show their strong convergence. We start by establishing new results on the asymptotic behavior of the Douglas-Rachford splitting algorithm for the sum of two operators. These results serve as a basis for the first algorithm. The second algorithm is based on an extension of a recent Dykstra-like method for computing the resolvent of the sum of two maximal monotone operators. Under standard qualification conditions, these two algorithms provide a means for computing the proximity operator of a weighted sum of lower semicontinuous convex functions. We show that a version of the second algorithm performs the same task without requiring any qualification condition. In turn, this provides a parallel splitting algorithm for qualification-free strongly convex programming. 2000 Mathematics Subject Classification: Primary 47H05; Secondary 47J25, 49M29, 65K05, 90C25.

[1]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[2]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[3]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[4]  Bernard Martinet,et al.  Algorithmes pour la résolution de problèmes d'optimisation et de minimax , 1972 .

[5]  H. Brezis.  Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .

[6]  Guy Pierra,et al.  Eclatement de Contraintes en Parallèle pour la Minimisation d'une Forme Quadratique , 1975, Optimization Techniques.

[7]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[8]  P. Lions,et al.  Une methode iterative de resolution d’une inequation variationnelle , 1978 .

[9]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[10]  Michel L. Lapidus,et al.  Generalization of the Trotter-Lie formula , 1981 .

[11]  Simeon Reich,et al.  A limit theorem for projections , 1983 .

[12]  J. Spingarn.  Partial inverse of a monotone operator , 1983 .

[13]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[14]  Guy Pierra,et al.  Decomposition through formalization in a product space , 1984, Math. Program..

[15]  Jonathan E. Spingarn,et al.  Applications of the method of partial inverses to convex programming: Decomposition , 1985, Math. Program..

[16]  F. Browder.  Nonlinear functional analysis and its applications , 1986 .

[17]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[18]  J. Spingarn.  A projection method for least-squares solutions to overdetermined systems of linear inequalities , 1987 .

[19]  J. Lawrence,et al.  On Fixed Points of Non-Expansive Piecewise Isometric Mappings , 1987 .

[20]  Christian Michelot,et al.  Applications and numerical convergence of the partial inverse method , 1988 .

[21]  E. Zeidler.  Nonlinear Functional Analysis and Its Applications: II/ A: Linear Monotone Operators , 1989 .

[22]  R. Mathar,et al.  A cyclic projection algorithm via duality , 1989 .

[23]  Alvaro R. De Pierro,et al.  On the convergence of Han's method for convex programming with quadratic objective , 1991, Math. Program..

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[26]  Patrick L. Combettes,et al.  Inconsistent signal feasibility problems: least-squares solutions in a product space , 1994, IEEE Trans. Signal Process..

[27]  Heinz H. Bauschke,et al.  Dykstra's Alternating Projection Algorithm for Two Sets , 1994 .

[28]  Tao Pham Dinh,et al.  Proximal Decomposition on the Graph of a Maximal Monotone Operator , 1995, SIAM J. Optim..

[29]  Heinz H. Bauschke,et al.  Legendre functions and the method of random Bregman projections , 1997 .

[30]  P. L. Combettes,et al.  Quasi-Fejérian Analysis of Some Optimization Algorithms , 2001 .

[31]  Heinz H. Bauschke,et al.  Finding best approximation pairs relative to two closed convex sets in Hilbert spaces , 2004, J. Approx. Theory.

[32]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[33]  Heinz H. Bauschke,et al.  The asymptotic behavior of the composition of two resolvents , 2005, Nonlinear Analysis: Theory, Methods & Applications.

[34]  Yair Censor,et al.  A multiprojection algorithm using Bregman projections in a product space , 1994, Numerical Algorithms.

[35]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[36]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[37]  P. L. Combettes,et al.  A Dykstra-like algorithm for two monotone operators , 2007 .

[38]  Valérie R. Wajs,et al.  A variational formulation for frame-based inverse problems , 2007 .

[39]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[40]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[41]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[42]  S. Simons.  From Hahn-Banach to monotonicity , 2008 .

[43]  Patrick L. Combettes,et al.  Visco-penalization of the sum of two monotone operators , 2008 .

[44]  H. Attouch,et al.  SOMME PONCTUELLE D’OPÉRATEURS MAXIMAUX MONOTONES , 2009 .