Convergence of a relaxed inertial proximal algorithm for maximally monotone operators

In a Hilbert space $${\mathcal {H}}$$ , given $$A{:}\;{\mathcal {H}}\rightarrow 2^{\mathcal {H}}$$ a maximally monotone operator, we study the convergence properties of a general class of relaxed inertial proximal algorithms. This study aims to extend to the case of the general monotone inclusion $$Ax \ni 0$$ the acceleration techniques initially introduced by Nesterov in the case of convex minimization. The relaxed form of the proximal algorithms plays a central role. It comes naturally with the regularization of the operator A by its Yosida approximation with a variable parameter, a technique recently introduced by Attouch–Peypouquet (Math Program Ser B, 2018. https://doi.org/10.1007/s10107-018-1252-x ) for a particular class of inertial proximal algorithms. Our study provides an algorithmic version of the convergence results obtained by Attouch–Cabot (J Differ Equ 264:7138–7182, 2018) in the case of continuous dynamical systems.

[1]  Z. Opial Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[2]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[3]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[4]  Radu Ioan Bot,et al.  Second Order Forward-Backward Dynamical Systems For Monotone Inclusion Problems , 2015, SIAM J. Control. Optim..

[5]  Dirk A. Lorenz,et al.  An Inertial Forward-Backward Algorithm for Monotone Inclusions , 2014, Journal of Mathematical Imaging and Vision.

[6]  F. Browder,et al.  Nonlinear ergodic theorems , 1976 .

[7]  Gregory B. Passty Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[8]  H. Brezis,et al.  Produits infinis de resolvantes , 1978 .

[9]  Patrick L. Combettes,et al.  Quasi-Nonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods , 2017, SIAM J. Optim..

[10]  H. Attouch,et al.  The heavy ball with friction dynamical system for convex constrained minimization problems , 2000 .

[11]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[12]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[13]  Hédy Attouch,et al.  Convergence Rates of Inertial Forward-Backward Algorithms , 2018, SIAM J. Optim..

[14]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[15]  H. Attouch,et al.  ASYMPTOTIC BEHAVIOR OF SECOND-ORDER DISSIPATIVE EVOLUTION EQUATIONS COMBINING POTENTIAL WITH NON-POTENTIAL EFFECTS ∗ , 2009, 0905.0092.

[16]  A. Cabot,et al.  Asymptotics for Some Proximal-like Method Involving Inertia and Memory Aspects , 2011 .

[17]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[18]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[19]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[20]  Michael C. Ferris,et al.  Operator-Splitting Methods for Monotone Affine Variational Inequalities, with a Parallel Application to Optimal Control , 1998, INFORMS J. Comput..

[21]  Juan Peypouquet,et al.  Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators , 2017, Mathematical Programming.

[22]  Julien M. Hendrickx,et al.  A generic online acceleration scheme for optimization algorithms via relaxation and inertia , 2016, Optim. Methods Softw..

[23]  P. Maingé Convergence theorems for inertial KM-type algorithms , 2008 .

[24]  H. Attouch,et al.  Convergence of damped inertial dynamics governed by regularized maximally monotone operators , 2018, Journal of Differential Equations.

[25]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[26]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..