Global Convergence to the Equilibrium of GANs using Variational Inequalities

In optimization, the negative gradient of a function denotes the direction of steepest descent. Furthermore, traveling in any direction orthogonal to the gradient maintains the value of the function. In this work, we show that these orthogonal directions that are ignored by gradient descent can be critical in equilibrium problems. Equilibrium problems have drawn heightened attention in machine learning due to the emergence of the Generative Adversarial Network (GAN). We use the framework of Variational Inequalities to analyze popular training algorithms for a fundamental GAN variant: the Wasserstein Linear-Quadratic GAN. We show that the steepest descent direction causes divergence from the equilibrium, and convergence to the equilibrium is achieved through following a particular orthogonal direction. We call this successful technique Crossing-the-Curl, named for its mathematical derivation as well as its intuition: identify the game's axis of rotation and move "across" space in the direction towards smaller "curling".

[1]  G. Stampacchia,et al.  On some non-linear elliptic differential-functional equations , 1966 .

[2]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[3]  Stella Dafermos,et al.  Traffic Equilibrium and Variational Inequalities , 1980 .

[4]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[5]  Stella Dafermos,et al.  An iterative scheme for variational inequalities , 1983, Math. Program..

[6]  Harris Drucker,et al.  Improving generalization performance using double backpropagation , 1992, IEEE Trans. Neural Networks.

[7]  A. Nagurney,et al.  Projected Dynamical Systems and Variational Inequalities with Applications , 1995 .

[8]  Jacques A. Ferland,et al.  Criteria for differentiable generalized monotone maps , 1996, Math. Program..

[9]  S. Schaible.  Generalized Monotone Nonsmooth Maps , 1996 .

[10]  M. Noor.  Generalized set-valued variational inequalities , 1998 .

[11]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[12]  Sean P. Meyn,et al.  The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning , 2000, SIAM J. Control. Optim..

[13]  Desmond J. Higham,et al.  Phase Space Error Control for Dynamical Systems , 1999, SIAM J. Sci. Comput..

[14]  Mauro Passacantando,et al.  Nash Equilibria, Variational Inequalities, and Dynamical Systems , 2002 .

[15]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[16]  Arkadi Nemirovski,et al.  Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems , 2004, SIAM J. Optim..

[17]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[18]  Geoffrey J. Gordon,et al.  No-regret learning in convex games , 2008, ICML '08.

[19]  V. Borkar.  Stochastic Approximation: A Dynamical Systems Viewpoint , 2008 .

[20]  A. Juditsky,et al.  Solving variational inequalities with Stochastic Mirror-Prox algorithm , 2008, 0809.0815.

[21]  Yishay Mansour,et al.  On the convergence of regret minimization dynamics in concave games , 2009, STOC '09.

[22]  T. Friesz.  Dynamic Optimization and Differential Games , 2010 .

[23]  Francisco Facchinei,et al.  Convex Optimization, Game Theory, and Variational Inequality Theory , 2010, IEEE Signal Processing Magazine.

[24]  T. Roughgarden,et al.  Intrinsic robustness of the price of anarchy , 2012, Commun. ACM.

[25]  John N. Tsitsiklis,et al.  NP-hardness of deciding convexity of quartic polynomials and related problems , 2010, Math. Program..

[26]  P. Olver.  Nonlinear Systems , 2013 .

[27]  Philip Thomas,et al.  GeNGA: A Generalization of Natural Gradient Ascent with Positive and Negative Convergence Results , 2014, ICML.

[28]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[29]  Angelia Nedic,et al.  Optimal robust smoothing extragradient algorithms for stochastic variational inequality problems , 2014, 53rd IEEE Conference on Decision and Control.

[30]  Bingsheng He,et al.  On the O(1/t) convergence rate of the projection and contraction methods for variational inequalities with Lipschitz continuous monotone operators , 2013, Computational Optimization and Applications.

[31]  Guanghui Lan,et al.  On the convergence properties of non-Euclidean extragradient methods for variational inequalities with generalized monotone operators , 2013, Comput. Optim. Appl..

[32]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[33]  Masatoshi Uehara,et al.  Generative Adversarial Nets from a Density Ratio Estimation Perspective , 2016, 1610.02920.

[34]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[35]  Sridhar Mahadevan,et al.  Online Monotone Optimization , 2016, ArXiv.

[36]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[37]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[38]  Vaibhava Goel,et al.  McGan: Mean and Covariance Feature Matching GAN , 2017, ICML.

[39]  J. Zico Kolter,et al.  Gradient descent GAN optimization is locally stable , 2017, NIPS.

[40]  Sridhar Mahadevan,et al.  Online Monotone Games , 2017, ArXiv.

[41]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[42]  Sebastian Nowozin,et al.  The Numerics of GANs , 2017, NIPS.

[43]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.

[44]  Yingyu Liang,et al.  Generalization and Equilibrium in Generative Adversarial Nets (GANs) , 2017, ICML.

[45]  Sebastian Nowozin,et al.  Stabilizing Training of Generative Adversarial Networks through Regularization , 2017, NIPS.

[46]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Luke de Oliveira,et al.  Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis , 2017, Computing and Software for Big Science.

[48]  Alfredo N. Iusem,et al.  Extragradient Method with Variance Reduction for Stochastic Variational Inequalities , 2017, SIAM J. Optim..

[49]  Fei Xia,et al.  Understanding GANs: the LQG Setting , 2017, ArXiv.

[50]  Tom Sercu,et al.  Fisher GAN , 2017, NIPS.

[51]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[52]  Thore Graepel,et al.  The Mechanics of n-Player Differentiable Games , 2018, ICML.

[53]  Sebastian Nowozin,et al.  Which Training Methods for GANs do actually Converge? , 2018, ICML.

[54]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[55]  Uday V. Shanbhag,et al.  Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants , 2014, Computational Optimization and Applications.

[56]  Gauthier Gidel,et al.  A Variational Inequality Perspective on Generative Adversarial Networks , 2018, ICLR.