Finding the Forward-Douglas–Rachford-Forward Method

We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas–Rachford and forward–backward–forward methods, respectively, solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas–Rachford and forward–backward–forward and show that it solves the 3-operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas–Rachford and forward–reflected–backward, a recently proposed alternative to forward–backward–forward by Malitsky and Tam (A forward–backward splitting method for monotone inclusions without cocoercivity, 2018. arXiv:1808.04162 ). We show that this second method solves the 3-operator problem generally, without further assumptions.

[1]  Mohamed-Jalal Fadili,et al.  A Generalized Forward-Backward Splitting , 2011, SIAM J. Imaging Sci..

[2]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[3]  Damek Davis,et al.  Forward-Backward-Half Forward Algorithm for Solving Monotone Inclusions , 2017, SIAM J. Optim..

[4]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[5]  Pontus Giselsson,et al.  Nonlinear Forward-Backward Splitting with Projection Correction , 2019, SIAM J. Optim..

[6]  Adrien B. Taylor,et al.  Operator Splitting Performance Estimation: Tight Contraction Factors and Optimal Parameter Selection , 2018, SIAM J. Optim..

[7]  Matthew K. Tam,et al.  A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity , 2018, SIAM J. Optim..

[8]  Hugo Raguet,et al.  A note on the forward-Douglas–Rachford splitting for monotone inclusion and convex optimization , 2017, Optim. Lett..

[9]  Luis M. Briceño-Arias,et al.  Forward–Partial Inverse–Forward Splitting for Solving Monotone Inclusions , 2014, J. Optim. Theory Appl..

[10]  Radu Ioan Bot,et al.  A Douglas-Rachford Type Primal-Dual Method for Solving Inclusions with Mixtures of Composite and Parallel-Sum Type Monotone Operators , 2012, SIAM J. Optim..

[11]  Patrick R. Johnstone,et al.  Projective splitting with forward steps only requires continuity , 2020, Optim. Lett..

[12]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[13]  Ronald E. Bruck.  On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space , 1977 .

[14]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[15]  Ernest K. Ryu,et al.  Scaled relative graphs: nonexpansive operators via 2D Euclidean geometry , 2019, Mathematical Programming.

[16]  Patrick R. Johnstone,et al.  Projective Splitting with Forward Steps: Asynchronous and Block-Iterative Operator Splitting , 2018, ArXiv.

[17]  Patrick L. Combettes,et al.  Systems of Structured Monotone Inclusions: Duality, Algorithms, and Applications , 2012, SIAM J. Optim..

[18]  RockaJellm.  MONOTONE OPERATORS ASSOCIATED WITH SADDLE . FUNCTIONS AND MINIMAX PROBLEMS R . 1 ' , 2022 .

[19]  Stephen P. Boyd,et al.  A Primer on Monotone Operator Methods , 2015 .

[20]  Panagiotis Patrinos,et al.  Asymmetric forward–backward–adjoint splitting for solving monotone inclusions involving three operators , 2016, Comput. Optim. Appl..

[21]  Adrien B. Taylor,et al.  Smooth strongly convex interpolation and exact worst-case performance of first-order methods , 2015, Mathematical Programming.

[22]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[23]  Ming Yan,et al.  A new primal-dual algorithm for minimizing the sum of three functions with a linear operator , 2016, 1611.09805.

[24]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[25]  Ernest K. Ryu.  Uniqueness of DRS as the 2 operator resolvent-splitting and impossibility of 3 operator resolvent-splitting , 2018, Math. Program..

[26]  Patrick L. Combettes,et al.  Warped proximal iterations for monotone inclusions , 2019, 1908.07077.

[27]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[28]  Marc Teboulle,et al.  Performance of first-order methods for smooth convex minimization: a novel approach , 2012, Mathematical Programming.

[29]  L. Briceño-Arias.  Forward-Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions , 2012, 1212.5942.

[30]  Patrick L. Combettes,et al.  Proximal Activation of Smooth Functions in Splitting Algorithms for Convex Minimization , 2018 .

[31]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .