Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization

The use of min-max optimization in adversarial training of deep neural network classifiers and training of generative adversarial networks has motivated the study of nonconvex-nonconcave optimization objectives, which frequently arise in these applications. Unfortunately, recent results have established that even approximate first-order stationary points of such objectives are intractable, even under smoothness conditions, motivating the study of min-max objectives with additional structure. We introduce a new class of structured nonconvex-nonconcave min-max optimization problems, proposing a generalization of the extragradient algorithm which provably converges to a stationary point. The algorithm applies not only to Euclidean spaces, but also to general $\ell_p$-normed finite-dimensional real vector spaces. We also discuss its stability under stochastic oracles and provide bounds on its sample complexity. Our iteration complexity and sample complexity bounds either match or improve the best known bounds for the same or less general nonconvex-nonconcave settings, such as those that satisfy variational coherence or in which a weak solution to the associated variational inequality problem is assumed to exist.

[1]  A. Gasnikov,et al.  Accelerated methods for composite non-bilinear saddle point problem , 2019, 1906.03620.

[2]  S. Shankar Sastry,et al.  On Gradient-Based Learning in Continuous Games , 2018, SIAM J. Math. Data Sci..

[3]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[4]  Nisheeth K. Vishnoi,et al.  A Second-order Equilibrium in Nonconvex-Nonconcave Min-max Optimization: Existence and Algorithm , 2020, ArXiv.

[5]  Patrick L. Combettes,et al.  Proximal Methods for Cohypomonotone Operators , 2004, SIAM J. Control. Optim..

[6]  Guodong Zhang,et al.  On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach , 2019, ICLR.

[7]  Constantinos Daskalakis,et al.  Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimization , 2018, ITCS.

[8]  Weiwei Kong,et al.  An Accelerated Inexact Proximal Point Method for Solving Nonconvex-Concave Min-Max Problems , 2019, SIAM J. Optim..

[9]  Noah Golowich,et al.  Last Iterate is Slower than Averaged Iterate in Smooth Convex-Concave Saddle Point Problems , 2020, COLT.

[10]  Michael I. Jordan,et al.  What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? , 2019, ICML.

[11]  Ioannis Mitliagkas,et al.  Negative Momentum for Improved Game Dynamics , 2018, AISTATS.

[12]  A. Banerjee.  Convex Analysis and Optimization , 2006 .

[13]  Christos H. Papadimitriou,et al.  On the Complexity of the Parity Argument and Other Inefficient Proofs of Existence , 1994, J. Comput. Syst. Sci..

[14]  Tengyuan Liang,et al.  Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks , 2018, AISTATS.

[15]  Zhengyuan Zhou,et al.  Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities , 2021, NeurIPS.

[16]  Renbo Zhao,et al.  Optimal Stochastic Algorithms for Convex-Concave Saddle-Point Problems , 2019 .

[17]  A. Nemirovski.  Regular Banach Spaces and Large Deviations of Random Sums , 2004 .

[18]  Yurii Nesterov,et al.  First-order methods of smooth convex optimization with inexact oracle , 2013, Mathematical Programming.

[19]  Jelena Diakonikolas.  Halpern Iteration for Near-Optimal and Parameter-Free Monotone Inclusion and Strong Solutions to Variational Inequalities , 2020, COLT.

[20]  Nisheeth K. Vishnoi,et al.  Greedy adversarial equilibrium: an efficient alternative to nonconvex-nonconcave min-max optimization , 2020, STOC.

[21]  J. Borwein,et al.  Uniformly convex functions on Banach spaces , 2008 .

[22]  Thomas Hofmann,et al.  Local Saddle Point Optimization: A Curvature Exploitation Approach , 2018, AISTATS.

[23]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[24]  Michael I. Jordan,et al.  Near-Optimal Algorithms for Minimax Optimization , 2020, COLT.

[25]  Karolin Papst,et al.  Techniques Of Variational Analysis , 2016 .

[26]  Stephen P. Boyd,et al.  Stochastic Mirror Descent in Variationally Coherent Optimization Problems , 2017, NIPS.

[27]  Christos H. Papadimitriou,et al.  Cycles in adversarial regularized learning , 2017, SODA.

[28]  Guanghui Lan,et al.  On the convergence properties of non-Euclidean extragradient methods for variational inequalities with generalized monotone operators , 2013, Comput. Optim. Appl..

[29]  Constantinos Daskalakis,et al.  The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization , 2018, NeurIPS.

[30]  Yurii Nesterov,et al.  Universal gradient methods for convex optimization problems , 2015, Math. Program..

[31]  Michael I. Jordan,et al.  On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems , 2019, ICML.

[32]  Ioannis Mitliagkas,et al.  A Tight and Unified Analysis of Extragradient for a Whole Spectrum of Differentiable Games , 2019, ArXiv.

[33]  Xiaodong Cui,et al.  Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets , 2020, ICLR.

[34]  Renbo Zhao.  Optimal Algorithms for Stochastic Three-Composite Convex-Concave Saddle Point Problems , 2019, 1903.01687.

[35]  Mingrui Liu,et al.  Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning , 2018, ArXiv.

[36]  Christos H. Papadimitriou,et al.  Exponential lower bounds for finding Brouwer fixed points , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[37]  Jacob Abernethy,et al.  Last-iterate convergence rates for min-max optimization , 2019, ArXiv.

[38]  Aryan Mokhtari,et al.  A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach , 2019, AISTATS.

[39]  Jason D. Lee,et al.  Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods , 2019, NeurIPS.

[40]  Constantinos Daskalakis,et al.  Training GANs with Optimism , 2017, ICLR.

[41]  Alexandre d'Aspremont,et al.  Optimal Affine-Invariant Smooth Minimization Algorithms , 2018, SIAM J. Optim..

[42]  Heinz H. Bauschke,et al.  Generalized monotone operators and their averaged resolvents , 2019, Mathematical Programming.

[43]  N. S. Aybat,et al.  A Primal-Dual Algorithm for General Convex-Concave Saddle Point Problems , 2018, 1803.01401.

[44]  Yura Malitsky,et al.  Golden ratio algorithms for variational inequalities , 2018, Mathematical Programming.

[45]  Mingrui Liu,et al.  Solving Weakly-Convex-Weakly-Concave Saddle-Point Problems as Successive Strongly Monotone Variational Inequalities , 2018 .

[46]  Niao He,et al.  Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems , 2020, ArXiv.

[47]  Constantinos Daskalakis,et al.  The complexity of constrained min-max optimization , 2020, STOC.

[48]  Prateek Jain,et al.  Efficient Algorithms for Smooth Minimax Optimization , 2019, NeurIPS.

[49]  Chuan-Sheng Foo,et al.  Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile , 2018, ICLR.

[50]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[51]  Nisheeth K. Vishnoi,et al.  A Provably Convergent and Practical Algorithm for Min-max Optimization with Applications to GANs , 2020, ArXiv.

[52]  Noah Golowich,et al.  Independent Policy Gradient Methods for Competitive Reinforcement Learning , 2021, NeurIPS.

[53]  D. Bertsekas.  Control of uncertain systems with a set-membership description of the uncertainty , 1971 .

[54]  Yongxin Chen,et al.  Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications , 2019, IEEE Transactions on Signal Processing.

[55]  Erfan Yazdandoost Hamedani,et al.  A Primal-Dual Algorithm with Line Search for General Convex-Concave Saddle Point Problems , 2020, SIAM J. Optim..