Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities

The optimization problems associated with training generative adversarial neural networks can be largely reduced to certain non-monotone variational inequality problems (VIPs), whereas existing convergence results are mostly based on monotone or strongly monotone assumptions. In this paper, we propose optimistic dual extrapolation (OptDE), a method that only performs one gradient evaluation per iteration. We show that OptDE is provably convergent to a strong solution under different coherent non-monotone assumptions. In particular, when a weak solution exists, the convergence rate of our method is O(1/✏2), which matches the best existing result of the methods with two gradient evaluations. Further, when a -weak solution exists, the convergence guarantee is improved to the linear rate O(log ✏ ). Along the way–as a byproduct of our inquiries into non-monotone variational inequalities–we provide the near-optimal O 1 ✏ log 1 ✏ convergence guarantee in terms of restricted strong merit function for monotone variational inequalities. We also show how our results can be naturally generalized to the stochastic setting, and obtain corresponding new convergence results. Taken together, our results contribute to the broad landscape of variational inequality–both non-monotone and monotone alike–by providing a novel and more practical algorithm with the state-of-the-art convergence guarantees.

[1]  Peter W. Glynn,et al.  Robust Power Management via Learning and Game Design , 2021, Oper. Res..

[2]  Yangyang Xu,et al.  Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems , 2018, Math. Program..

[3]  Jelena Diakonikolas.  Halpern Iteration for Near-Optimal and Parameter-Free Monotone Inclusion and Strong Solutions to Variational Inequalities , 2020, COLT.

[4]  Noah Golowich,et al.  Last Iterate is Slower than Averaged Iterate in Smooth Convex-Concave Saddle Point Problems , 2020, COLT.

[5]  Mingrui Liu,et al.  Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets , 2019, ICLR.

[6]  Wei Peng,et al.  Training GANs with centripetal acceleration , 2019, Optim. Methods Softw..

[7]  Stephen P. Boyd,et al.  On the Convergence of Mirror Descent beyond Stochastic Convex Programming , 2017, SIAM J. Optim..

[8]  J. Malick,et al.  On the convergence of single-call stochastic extra-gradient methods , 2019, NeurIPS.

[9]  Jason D. Lee,et al.  Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods , 2019, NeurIPS.

[10]  Gauthier Gidel,et al.  A Variational Inequality Perspective on Generative Adversarial Networks , 2018, ICLR.

[11]  Tengyuan Liang,et al.  Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks , 2018, AISTATS.

[12]  Zhengyuan Zhou,et al.  Learning in games with continuous action sets and unknown payoff functions , 2016, Mathematical Programming.

[13]  Uday V. Shanbhag,et al.  Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants , 2014, Computational Optimization and Applications.

[14]  Mingrui Liu,et al.  Solving Weakly-Convex-Weakly-Concave Saddle-Point Problems as Weakly-Monotone Variational Inequality , 2018 .

[15]  Constantinos Daskalakis,et al.  Training GANs with Optimism , 2017, ICLR.

[16]  Peter W. Glynn,et al.  Mirror descent learning in continuous games , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[17]  Stephen P. Boyd,et al.  Stochastic Mirror Descent in Variationally Coherent Optimization Problems , 2017, NIPS.

[18]  Yuanzhi Li,et al.  Convergence Analysis of Two-layer Neural Networks with ReLU Activation , 2017, NIPS.

[19]  Alfredo N. Iusem,et al.  Extragradient Method with Variance Reduction for Stochastic Variational Inequalities , 2017, SIAM J. Optim..

[20]  Uday V. Shanbhag,et al.  On the analysis of reflected gradient and splitting methods for monotone stochastic variational inequality problems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[21]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[22]  Yu. V. Malitsky,et al.  Projected Reflected Gradient Methods for Monotone Variational Inequalities , 2015, SIAM J. Optim..

[23]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[24]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[25]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[26]  Guanghui Lan,et al.  On the convergence properties of non-Euclidean extragradient methods for variational inequalities with generalized monotone operators , 2013, Comput. Optim. Appl..

[27]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[28]  Karthik Sridharan,et al.  Online Learning with Predictable Sequences , 2012, COLT.

[29]  Rong Jin,et al.  25th Annual Conference on Learning Theory Online Optimization with Gradual Variations , 2022 .

[30]  Siegfried Schaible,et al.  Pseudomonotone Operators: A Survey of the Theory and Its Applications , 2012, J. Optim. Theory Appl..

[31]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[32]  Lin Xiao,et al.  Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization , 2009, J. Mach. Learn. Res..

[33]  Maximization of the lift/drag ratio of airfoils with a turbulent boundary layer: Sharp estimates, approximation, and numerical solutions , 2009 .

[34]  Yurii Nesterov,et al.  Dual extrapolation and its applications to solving variational inequalities and related problems , 2003, Math. Program..

[35]  A. Rousseau,et al.  Trade-off between Fuel Economy and Cost for Advanced Vehicle Configurations , 2007 .

[36]  Arkadi Nemirovski,et al.  Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems , 2004, SIAM J. Optim..

[37]  Luigi Brighi,et al.  Characterizations of pseudomonotone maps and economic equilibrium , 2002 .

[38]  K. Ball,et al.  Sharp uniform convexity and smoothness inequalities for trace norms , 1994 .

[39]  W. DeSarbo,et al.  Product Positioning Under Price Competition , 1990 .

[40]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[41]  L. Popov.  A modification of the Arrow-Hurwicz method for search of saddle points , 1980 .

[42]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[43]  S. Karamardian.  Complementarity problems over cones with monotone and pseudomonotone maps , 1976 .

[44]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .