Iterative regularization methods with new stepsize rules for solving variational inclusions

The paper concerns with three iterative regularization methods for solving a variational inclusion problem of the sum of two operators, the one is maximally monotone and the another is monotone and Lipschitz continuous, in a Hilbert space. We first describe how to incorporate regularization terms in the methods of forward-backward types, and then establish the strong convergence of the resulting methods. With several new stepsize rules considered, the methods can work with or without knowing previously the Lipschitz constant of cost operator. Unlike known hybrid methods, the strong convergence of the proposed methods comes from the regularization technique. Several applications to signal recovery problems and optimal control problems together with numerical experiments are also presented in this paper. Our numerical results have illustrated the fast convergence and computational effectiveness of the new methods over known hybrid methods.

[1]  L. Muu,et al.  Modified forward–backward splitting method for variational inclusions , 2020, 4OR.

[2]  G. Stampacchia,et al.  On some non-linear elliptic differential-functional equations , 1966 .

[3]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[4]  Jen-Chih Yao,et al.  Pseudo-monotone complementarity problems in Hilbert space , 1992 .

[5]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[6]  Yair Censor,et al.  A multiprojection algorithm using Bregman projections in a product space , 1994, Numerical Algorithms.

[7]  W. Takahashi,et al.  Strong Convergence Theorems for Maximal Monotone Operators with Nonlinear Mappings in Hilbert Spaces , 2010, J. Optim. Theory Appl..

[8]  Hong-Kun Xu,et al.  Averaged Mappings and the Gradient-Projection Algorithm , 2011, J. Optim. Theory Appl..

[9]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[11]  Pham Kim Quy,et al.  Three-operator splitting algorithm for a class of variational inclusion problems , 2020, Bulletin of the Iranian Mathematical Society.

[12]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[13]  Pham Ky Anh,et al.  Modified extragradient-like algorithms with new stepsizes for variational inequalities , 2019, Computational Optimization and Applications.

[14]  José Yunier Bello Cruz,et al.  On the convergence of the forward–backward splitting method with linesearches , 2015, Optim. Methods Softw..

[15]  Stephen J. Wright,et al.  Optimization for Machine Learning , 2013 .

[16]  Y. Cho,et al.  Modified Extragradient Method for Pseudomonotone Variational Inequalities in Infinite Dimensional Hilbert Spaces , 2020, Vietnam Journal of Mathematics.

[17]  Ernest K. Ryu,et al.  Finding the Forward-Douglas–Rachford-Forward Method , 2019, J. Optim. Theory Appl..

[18]  高橋 渉.  Nonlinear functional analysis : fixed point theory and its applications , 2000 .

[19]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[20]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[21]  S. Reich,et al.  Two Bregman projection methods for solving variational inequalities , 2020, Optimization.

[22]  Elena V. Khoroshilova,et al.  Extragradient-type method for optimal control problem with linear constraints and convex objective function , 2013, Optim. Lett..

[23]  Hong-Ku N Xu.  Another control condition in an iterative method for nonexpansive mappings , 2002, Bulletin of the Australian Mathematical Society.

[24]  Y. Shehu.  Convergence Results of Forward-Backward Algorithms for Sum of Monotone Operators in Banach Spaces , 2019, Results in Mathematics.

[25]  V. Cevher,et al.  A Reflected Forward-Backward Splitting Method for Monotone Inclusions Involving Lipschitzian Operators , 2019, Set-Valued and Variational Analysis.

[26]  Le Dung Muu,et al.  An Explicit Extragradient Algorithm for Solving Variational Inequalities , 2020, J. Optim. Theory Appl..

[27]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[28]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[29]  P. Cholamjiak,et al.  Modified extragradient method with Bregman distance for variational inequalities , 2020, Applicable Analysis.

[30]  Martin Seydenschwanz,et al.  Convergence results for the discrete regularization of linear-quadratic control problems with bang–bang solutions , 2015, Comput. Optim. Appl..

[31]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .