Convex-Concave Min-Max Stackelberg Games

Min-max optimization problems (i.e., min-max games) have been attracting a great deal of attention because of their applicability to a wide range of machine learning problems. Although significant progress has been made recently, the literature to date has focused on games with independent strategy sets; little is known about solving games with dependent strategy sets, which can be characterized as min-max Stackelberg games. We introduce two first-order methods that solve a large class of convex-concave min-max Stackelberg games, and show that our methods converge in polynomial time. Min-max Stackelberg games were first studied by Wald, under the posthumous name of Wald’s maximin model, a variant of which is the main paradigm used in robust optimization, which means that our methods can likewise solve many convex robust optimization problems. We observe that the computation of competitive equilibria in Fisher markets also comprises a min-max Stackelberg game. Further, we demonstrate the efficacy and efficiency of our algorithms in practice by computing competitive equilibria in Fisher markets with varying utility structures. Our experiments suggest potential ways to extend our theoretical results, by demonstrating how different smoothness properties can affect the convergence rate of our algorithms.

[1]  Meisam Razaviyayn,et al.  Efficient Search of First-Order Nash Equilibria in Nonconvex-Concave Smooth Min-Max Problems , 2021, SIAM J. Optim..

[2]  H. Stackelberg,et al.  Marktform und Gleichgewicht , 1935 .

[3]  Kevin Waugh,et al.  Solving Large Extensive-Form Games with Strategy Constraints , 2018, AAAI.

[4]  P. Tseng On linear convergence of iterative methods for the variational inequality problem , 1995 .

[5]  Le Song,et al.  SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation , 2017, ICML.

[6]  T. C. Oshima,et al.  Type I Error Rates for Welch’s Test and James’s Second-Order Test Under Nonnormality and Inequality of Variance When There Are Two Groups , 1994 .

[7]  J. Danskin The Theory of Max-Min, with Applications , 1966 .

[8]  Hamed Kebriaei,et al.  Discrete-Time Robust Hierarchical Linear-Quadratic Dynamic Games , 2018, IEEE Transactions on Automatic Control.

[9]  Liwei Zhang,et al.  Optimality Conditions for Constrained Minimax Optimization , 2020, CSIAM Transactions on Applied Mathematics.

[10]  Yangyang Xu,et al.  Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems , 2018, Math. Program..

[11]  Jason D. Lee,et al.  Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods , 2019, NeurIPS.

[12]  Lu Zhang,et al.  FairGAN: Fairness-aware Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[13]  H. Scarf,et al.  How to Compute Equilibrium Prices in 1891 , 2005 .

[14]  Arkadi Nemirovski,et al.  Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems , 2004, SIAM J. Optim..

[15]  Renbo Zhao Optimal Algorithms for Stochastic Three-Composite Convex-Concave Saddle Point Problems , 2019, 1903.01687.

[16]  John C. Duchi,et al.  Certifying Some Distributional Robustness with Principled Adversarial Training , 2017, ICLR.

[17]  E. Aiyoshi,et al.  Necessary conditions for min-max problems and algorithms by a relaxation procedure , 1980 .

[18]  N. S. Aybat,et al.  A Primal-Dual Algorithm for General Convex-Concave Saddle Point Problems , 2018, 1803.01401.

[19]  Amos J. Storkey,et al.  Censoring Representations with an Adversary , 2015, ICLR.

[20]  Michael I. Jordan,et al.  On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems , 2019, ICML.

[21]  Renato A. Krohling,et al.  A differential evolution approach for solving constrained min-max optimization problems , 2012, Expert Syst. Appl..

[22]  Mofidul Islam A First Course in Real Analysis , 2013 .

[23]  A. Smithies The Stability of Competitive Equilibrium , 1942 .

[24]  L. Walras Elements of Pure Economics, or The Theory of Social Wealth , 1955 .

[25]  A Primal Dual Smoothing Framework for Max-Structured Nonconvex Optimization , 2020, 2003.04375.

[26]  Sergio Grammatico,et al.  Local Stackelberg Equilibrium Seeking in Generalized Aggregative Games , 2020, IEEE Transactions on Automatic Control.

[27]  Michael I. Jordan,et al.  What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? , 2019, ICML.

[28]  Shimrit Shtern,et al.  First-order algorithms for robust optimization problems via convex-concave saddle-point Lagrangian reformulation , 2021, 2101.02669.

[29]  Paul R. Milgrom,et al.  Envelope Theorems for Arbitrary Choice Sets , 2002 .

[30]  Shie Mannor,et al.  Oracle-Based Robust Optimization via Online Learning , 2014, Oper. Res..

[31]  Kohske Takahashi,et al.  Welcome to the Tidyverse , 2019, J. Open Source Softw..

[32]  Yixin Tao,et al.  Balancing the Robustness and Convergence of Tatonnement , 2019, ArXiv.

[33]  Gauthier Gidel,et al.  A Variational Inequality Perspective on Generative Adversarial Networks , 2018, ICLR.

[34]  Yurii Nesterov,et al.  Dual extrapolation and its applications to solving variational inequalities and related problems , 2007, Math. Program..

[35]  Toniann Pitassi,et al.  Learning Adversarially Fair and Transferable Representations , 2018, ICML.

[36]  Nikhil R. Devanur,et al.  Market equilibrium via a primal-dual-type algorithm , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[37]  Ioannis Mitliagkas,et al.  Lower Bounds and Conditioning of Differentiable Games , 2019, ArXiv.

[38]  Yurii Nesterov,et al.  Solving Strongly Monotone Variational and Quasi-Variational Inequalities , 2006 .

[39]  K. Jarrod Millman,et al.  Array programming with NumPy , 2020, Nat..

[40]  S. Berberian A first course in real analysis , 1994 .

[41]  Prateek Jain,et al.  Efficient Algorithms for Smooth Minimax Optimization , 2019, NeurIPS.

[42]  Le Song,et al.  Kernel Exponential Family Estimation via Doubly Dual Embedding , 2018, AISTATS.

[43]  Songtao Lu,et al.  Block Alternating Optimization for Non-convex Min-max Problems: Algorithms and Applications in Signal Processing and Communications , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[45]  Tim Roughgarden,et al.  Algorithmic Game Theory , 2007 .

[46]  Michael I. Jordan,et al.  Near-Optimal Algorithms for Minimax Optimization , 2020, COLT.

[47]  René Henrion,et al.  On constraint qualifications , 1992 .

[48]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[49]  Tuomas Sandholm,et al.  Regret Circuits: Composability of Regret Minimizers , 2018, ICML.

[50]  J. Neumann Zur Theorie der Gesellschaftsspiele , 1928 .

[51]  Nam Ho-Nguyen,et al.  Online First-Order Framework for Robust Convex Optimization , 2016, Oper. Res..

[52]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[53]  Yongxin Chen,et al.  On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator , 2019, ArXiv.

[54]  E. Aiyoshi,et al.  A new computational method for Stackelberg and min-max problems by use of a penalty method , 1981 .

[55]  Nikhil R. Devanur,et al.  Tatonnement beyond gross substitutes?: gradient descent to the rescue , 2013, STOC '13.

[56]  Jason D. Lee,et al.  On the Convergence and Robustness of Training GANs with Regularized Optimal Transport , 2018, NeurIPS.

[57]  S. Afriat Theory of Maxima and the Method of Lagrange , 1971 .

[58]  Francisco Facchinei,et al.  Generalized Nash Equilibrium Problems , 2010, Ann. Oper. Res..

[59]  Constantinos Daskalakis,et al.  The complexity of constrained min-max optimization , 2020, STOC.

[60]  A. Wald Statistical Decision Functions Which Minimize the Maximum Risk , 1945 .

[61]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[62]  K. Arrow,et al.  EXISTENCE OF AN EQUILIBRIUM FOR A COMPETITIVE ECONOMY , 1954 .

[63]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[64]  Vijay V. Vazirani,et al.  Market equilibria for homothetic, quasi-concave utilities and economies of scale in production , 2005, SODA '05.

[65]  L. Shapley,et al.  Potential Games , 1994 .

[66]  Mingrui Liu,et al.  Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning , 2018, ArXiv.

[67]  Yuan Gao,et al.  First-order methods for large-scale market equilibrium computation , 2020, NeurIPS.

[68]  C. Berge Topological Spaces: including a treatment of multi-valued functions , 2010 .

[69]  A. Gasnikov,et al.  Accelerated methods for composite non-bilinear saddle point problem , 2019, 1906.03620.

[70]  E. Eisenberg,et al.  CONSENSUS OF SUBJECTIVE PROBABILITIES: THE PARI-MUTUEL METHOD, , 1959 .

[71]  N. S. Aybat,et al.  Iteration Complexity of Randomized Primal-Dual Methods for Convex-Concave Saddle Point Problems , 2018 .

[72]  Enrique Areyan Viqueira,et al.  Tâtonnement Beyond Constant Elasticity of Substitution , 2021, ArXiv.

[73]  Paul Dütting,et al.  Optimal auctions through deep learning , 2017, ICML.