A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity

In this work, we propose a simple modification of the forward-backward splitting method for finding a zero in the sum of two monotone operators. Our method converges under the same assumptions as Tseng's forward-backward-forward method, namely, it does not require cocoercivity of the single-valued operator. Moreover, each iteration only requires one forward evaluation rather than two as is the case for Tseng's method. Variants of the method incorporating a linesearch, relaxation and inertia, or a structured three operator inclusion are also discussed.

[1]  H. Attouch,et al.  Convergence of a Relaxed Inertial Forward–Backward Algorithm for Structured Monotone Inclusions , 2019, Applied Mathematics & Optimization.

[2]  H. Robbins,et al.  A Convergence Theorem for Non Negative Almost Supermartingales and Some Applications , 1985 .

[3]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[4]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[5]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[6]  R. Díaz Millán,et al.  A variant of forward-backward splitting method for the sum of two monotone operators with a new search strategy , 2015 .

[7]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[8]  Patrick L. Combettes,et al.  Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping , 2014 .

[9]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[10]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[11]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[12]  Benar Fux Svaiter,et al.  On Weak Convergence of the Douglas-Rachford Method , 2010, SIAM J. Control. Optim..

[13]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[14]  Damek Davis,et al.  Forward-Backward-Half Forward Algorithm for Solving Monotone Inclusions , 2017, SIAM J. Optim..

[15]  Patrick R. Johnstone,et al.  Projective Splitting with Forward Steps: Asynchronous and Block-Iterative Operator Splitting , 2018, ArXiv.

[16]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[17]  R. Tyrrell Rockafellar,et al.  Convergence Rates in Forward-Backward Splitting , 1997, SIAM J. Optim..

[18]  Chuan-Sheng Foo,et al.  Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile , 2018, ICLR.

[19]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[20]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[21]  Kun Yuan,et al.  ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems and GANs , 2019, ArXiv.

[22]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[23]  Panayotis Mertikopoulos,et al.  On the convergence of single-call stochastic extra-gradient methods , 2019, NeurIPS.

[24]  P. L. Combettes,et al.  Quasi-Fejérian Analysis of Some Optimization Algorithms , 2001 .

[25]  Dirk A. Lorenz,et al.  An Inertial Forward-Backward Algorithm for Monotone Inclusions , 2014, Journal of Mathematical Imaging and Vision.

[26]  Matthew K. Tam,et al.  Shadow Douglas–Rachford Splitting for Monotone Inclusions , 2019, Applied Mathematics & Optimization.

[27]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[28]  N. S. Aybat,et al.  A Primal-Dual Algorithm for General Convex-Concave Saddle Point Problems , 2018, 1803.01401.

[29]  Gauthier Gidel,et al.  A Variational Inequality Perspective on Generative Adversarial Networks , 2018, ICLR.

[30]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[31]  Radu Ioan Bot,et al.  A Primal-Dual Splitting Algorithm for Finding Zeros of Sums of Maximal Monotone Operators , 2012, SIAM J. Optim..

[32]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[33]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[34]  Yura Malitsky,et al.  Golden ratio algorithms for variational inequalities , 2018, Mathematical Programming.