Forcing strong convergence of proximal point iterations in a Hilbert space

Abstract.This paper concerns with convergence properties of the classical proximal point algorithm for finding zeroes of maximal monotone operators in an infinite-dimensional Hilbert space. It is well known that the proximal point algorithm converges weakly to a solution under very mild assumptions. However, it was shown by Güler [11] that the iterates may fail to converge strongly in the infinite-dimensional case. We propose a new proximal-type algorithm which does converge strongly, provided the problem has a solution. Moreover, our algorithm solves proximal point subproblems inexactly, with a constructive stopping criterion introduced in [31]. Strong convergence is forced by combining proximal point iterations with simple projection steps onto intersection of two halfspaces containing the solution set. Additional cost of this extra projection step is essentially negligible since it amounts, at most, to solving a linear system of two equations in two unknowns.

[1]  Osman Güer.  On the convergence of the proximal point algorithm for convex minimization , 1991 .

[2]  Heinz H. Bauschke,et al.  A Weak-to-Strong Convergence Principle for Fejé-Monotone Methods in Hilbert Spaces , 2001, Math. Oper. Res..

[3]  B. Martinet,et al.  R'egularisation d''in'equations variationnelles par approximations successives , 1970 .

[4]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[5]  E. H. Zarantonello.  Projections on Convex Sets in Hilbert Space and Spectral Theory: Part I. Projections on Convex Sets: Part II. Spectral Theory , 1971 .

[6]  Jonathan Eckstein,et al.  Approximate iterations in Bregman-function-based proximal algorithms , 1998, Math. Program..

[7]  H. Brezis,et al.  Produits infinis de resolvantes , 1978 .

[8]  J. Aplevich,et al.  Lecture Notes in Control and Information Sciences , 1979 .

[9]  Benar Fux Svaiter,et al.  An Inexact Hybrid Generalized Proximal Point Algorithm and Some New Results on the Theory of Bregman Functions , 2000, Math. Oper. Res..

[10]  F. Giannessi,et al.  Nonlinear Optimization and Related Topics , 2000 .

[11]  M. Solodov,et al.  A Hybrid Approximate Extragradient – Proximal Point Algorithm Using the Enlargement of a Maximal Monotone Operator , 1999 .

[12]  Manfred Laumen.  Newton's Mesh Independence Principle for a Class Of Optimal Shape Design Problems , 1999 .

[13]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[14]  J. Burke,et al.  A Variable Metric Proximal Point Algorithm for Monotone Operators , 1999 .

[15]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[16]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[17]  S. Reich.  Weak convergence theorems for nonexpansive mappings in Banach spaces , 1979 .

[18]  Patrick L. Combettes,et al.  Strong Convergence of Block-Iterative Outer Approximation Methods for Convex Optimization , 2000, SIAM J. Control. Optim..

[19]  F. Luque.  Asymptotic convergence analysis of the proximal point algorithm , 1984 .

[20]  M. V. Solodovy,et al.  A Hybrid Projection{proximal Point Algorithm , 1998 .

[21]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[22]  E. Allgower,et al.  A mesh-independence principle for operator equations and their discretizations , 1986 .

[23]  Osman Güler,et al.  New Proximal Point Algorithms for Convex Minimization , 1992, SIAM J. Optim..

[24]  Benar Fux Svaiter,et al.  A Truly Globally Convergent Newton-Type Method for the Monotone Nonlinear Complementarity Problem , 1999, SIAM J. Optim..

[25]  Michael C. Ferris,et al.  Finite termination of the proximal point algorithm , 1991, Math. Program..

[26]  R. Rockafellar.  On the maximality of sums of nonlinear monotone operators , 1970 .

[27]  E. Allgower,et al.  Application of the mesh independence principle to mesh refinement , 1987 .

[28]  J. Frédéric Bonnans,et al.  A family of variable metric proximal methods , 1995, Math. Program..

[29]  Mikhail V. Solodov,et al.  A Globally Convergent Inexact Newton Method for Systems of Monotone Equations , 1998 .

[30]  M. Solodov,et al.  A New Projection Method for Variational Inequality Problems , 1999 .

[31]  G. Minty.  Monotone (nonlinear) operators in Hilbert space , 1962 .

[32]  M. Solodov,et al.  A Comparison of Rates of Convergence of Two Inexact Proximal Point Algorithms , 2000 .