Golden ratio algorithms for variational inequalities

The paper presents a fully explicit algorithm for monotone variational inequalities. The method uses variable stepsizes that are computed using two previous iterates as an approximation of the local Lipschitz constant without running a linesearch. Thus, each iteration of the method requires only one evaluation of a monotone operator $F$ and a proximal mapping $g$. The operator $F$ need not be Lipschitz-continuous, which also makes the algorithm interesting in the area of composite minimization where one cannot use the descent lemma. The method exhibits an ergodic $O(1/k)$ convergence rate and $R$-linear rate, if $F, g$ satisfy the error bound condition. We discuss possible applications of the method to fixed point problems. We discuss possible applications of the method to fixed point problems as well as its different generalizations.

[1]  Volkan Cevher,et al.  Composite self-concordant minimization , 2013, J. Mach. Learn. Res..

[2]  Hongwei Liu,et al.  A Modified Projected Gradient Method for Monotone Variational Inequalities , 2018, J. Optim. Theory Appl..

[3]  V. Berinde.  Iterative Approximation of Fixed Points , 2007 .

[4]  Yurii Nesterov,et al.  Dual extrapolation and its applications to solving variational inequalities and related problems , 2003, Math. Program..

[5]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[6]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[7]  I. Konnov.  A Class of Combined Iterative Methods for Solving Variational Inequalities , 1997 .

[8]  Yair Censor,et al.  The Subgradient Extragradient Method for Solving Variational Inequalities in Hilbert Space , 2011, J. Optim. Theory Appl..

[9]  Patrick T. Harker,et al.  A variational inequality approach for the determination of oligopolistic market equilibrium , 1984, Math. Program..

[10]  M. Solodov,et al.  A New Projection Method for Variational Inequality Problems , 1999 .

[11]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[12]  A. Iusem,et al.  A variant of korpelevich’s method for variational inequalities with a new search strategy , 1997 .

[13]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[14]  A. Juditsky,et al.  Solving variational inequalities with Stochastic Mirror-Prox algorithm , 2008, 0809.0815.

[15]  Yurii Nesterov,et al.  Relatively Smooth Convex Optimization by First-Order Methods, and Applications , 2016, SIAM J. Optim..

[16]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[17]  Yu Malitsky,et al.  Proximal extrapolated gradient methods for variational inequalities , 2016, Optim. Methods Softw..

[18]  O. SIAMJ.,et al.  PROX-METHOD WITH RATE OF CONVERGENCE O(1/t) FOR VARIATIONAL INEQUALITIES WITH LIPSCHITZ CONTINUOUS MONOTONE OPERATORS AND SMOOTH CONVEX-CONCAVE SADDLE POINT PROBLEMS∗ , 2004 .

[19]  Panagiotis Patrinos,et al.  SuperMann: A Superlinearly Convergent Algorithm for Finding Fixed Points of Nonexpansive Operators , 2016, IEEE Transactions on Automatic Control.

[20]  Antonin Chambolle,et al.  On the ergodic convergence rates of a first-order primal–dual algorithm , 2016, Math. Program..

[21]  S. Ishikawa.  Fixed points by a new iteration method , 1974 .

[22]  Hanif D. Sherali,et al.  A mathematical programming approach for determining oligopolistic market equilibrium , 1982, Math. Program..

[23]  Radu Ioan Bot,et al.  A forward-backward-forward differential equation and its asymptotic properties , 2015, 1503.07728.

[24]  Stephen P. Boyd,et al.  A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights , 2014, J. Mach. Learn. Res..

[25]  Juan Peypouquet,et al.  Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity , 2018, Math. Program..

[26]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[27]  Marc Teboulle,et al.  A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications , 2017, Math. Oper. Res..

[28]  P. L. Combettes,et al.  The Convex Feasibility Problem in Image Recovery , 1996 .

[29]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[30]  Yu. V. Malitsky,et al.  Projected Reflected Gradient Methods for Monotone Variational Inequalities , 2015, SIAM J. Optim..

[31]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[32]  Liu Qihou,et al.  On Naimpally and Singh's open questions , 1987 .

[33]  H. Attouch,et al.  A Dynamical Approach to Convex Minimization Coupling Approximation with the Steepest Descent Method , 1996 .

[34]  Dirk A. Lorenz,et al.  An Inertial Forward-Backward Algorithm for Monotone Inclusions , 2014, Journal of Mathematical Imaging and Vision.

[35]  Arkadi S. Nemirovsky,et al.  Information-based complexity of linear operator equations , 1992, J. Complex..

[36]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[37]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[38]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[39]  Arkadi Nemirovski,et al.  A Randomized Mirror-Prox Method for Solving Structured Large-Scale Matrix Saddle-Point Problems , 2011, SIAM J. Optim..

[40]  Boris Polyak.  Some methods of speeding up the convergence of iteration methods , 1964 .

[41]  Mikhail V. Solodov.  Convergence rate analysis of iteractive algorithms for solving variational inequality problems , 2003, Math. Program..

[42]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[43]  R. Tyrrell Rockafellar,et al.  Convergence Rates in Forward-Backward Splitting , 1997, SIAM J. Optim..

[44]  P. Tseng.  On linear convergence of iterative methods for the variational inequality problem , 1995 .

[45]  S. I. Lyashko,et al.  Low-cost modification of Korpelevich’s methods for monotone equilibrium problems , 2011 .

[46]  Yu. V. Malitsky,et al.  An Extragradient Algorithm for Monotone Variational Inequalities , 2014 .

[47]  Bingsheng He,et al.  A new method for a class of linear variational inequalities , 1994, Math. Program..

[48]  Stephen P. Boyd,et al.  Line search for averaged operator iteration , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[49]  Renato D. C. Monteiro,et al.  Complexity of Variants of Tseng's Modified F-B Splitting and Korpelevich's Methods for Hemivariational Inequalities with Applications to Saddle-point and Convex Optimization Problems , 2011, SIAM J. Optim..

[50]  Jong-Shi Pang,et al.  Error bounds in mathematical programming , 1997, Math. Program..

[51]  R. Díaz Millán,et al.  A variant of forward-backward splitting method for the sum of two monotone operators with a new search strategy , 2015 .

[52]  S. Naimpally,et al.  Extensions of some fixed point theorems of Rhoades , 1983 .

[53]  E. Khobotov.  Modification of the extra-gradient method for solving variational inequalities and certain optimization problems , 1989 .

[54]  D. Russell Luke,et al.  Quantitative Convergence Analysis of Iterated Expansive, Set-Valued Mappings , 2016, Math. Oper. Res..

[55]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[56]  L. Popov.  A modification of the Arrow-Hurwicz method for search of saddle points , 1980 .

[57]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.