PAC Learnability of Approximate Nash Equilibrium in Bimatrix Games

Computing Nash equilibrium in bimatrix games is PPAD-hard, and many works have focused on the approximate solutions. When games are generated from a fixed unknown distribution, learning a Nash predictor via data-driven approaches can be preferable. In this paper, we study the learnability of approximate Nash equilibrium in bimatrix games. We prove that Lipschitz function class is agnostic Probably Approximately Correct (PAC) learnable with respect to Nash approximation loss. Additionally, to demonstrate the advantages of learning a Nash predictor, we develop a model that can efficiently approximate solutions for games under the same distribution. We show by experiments that the solutions from our Nash predictor can serve as effective initializing points for other Nash solvers.

[1]  Ioannis Ch. Paschalidis,et al.  Data-driven estimation of travel latency cost functions via inverse optimization in multi-class transportation networks , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[2]  L. Shapley,et al.  Fictitious Play Property for Games with Identical Interests , 1996 .

[3]  Xiaotie Deng,et al.  On Tightness of the Tsaknakis-Spirakis Algorithm for Approximate Nash Equilibrium , 2021, SAGT.

[4]  John Fearnley,et al.  Distributed Methods for Computing Approximate Equilibria , 2015, Algorithmica.

[5]  Nicolò Cesa-Bianchi,et al.  Gambling in a rigged casino: The adversarial multi-armed bandit problem , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[6]  Xidong Feng,et al.  Discovering Multi-Agent Auto-Curricula in Two-Player Zero-Sum Games , 2021, ArXiv.

[7]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Vishal Gupta,et al.  Data-driven estimation in equilibrium using inverse optimization , 2013, Mathematical Programming.

[9]  Panagiota N. Panagopoulou,et al.  Polynomial algorithms for approximating Nash equilibria of bimatrix games , 2006, Theor. Comput. Sci..

[10]  Aranyak Mehta,et al.  A note on approximate Nash equilibria , 2006, Theor. Comput. Sci..

[11]  V. Conitzer,et al.  Approximation guarantees for fictitious play , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[12]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[13]  Yuhao Li,et al.  On the Complexity of Computing Markov Perfect Equilibrium in General-Sum Stochastic Games , 2021, Electron. Colloquium Comput. Complex..

[14]  Xiaotie Deng,et al.  Settling the complexity of computing two-player Nash equilibria , 2007, JACM.

[15]  Evangelos Markakis,et al.  New algorithms for approximate Nash equilibria in bimatrix games , 2010, Theor. Comput. Sci..

[16]  Nicola Gatti,et al.  Learning Probably Approximately Correct Maximin Strategies in Simulation-Based Games with Infinite Strategy Spaces , 2020, AAMAS.

[17]  J. Zico Kolter,et al.  What game are we playing? End-to-end learning in normal and extensive form games , 2018, IJCAI.

[18]  Yoav Shoham,et al.  Run the GAMUT: a comprehensive approach to evaluating game-theoretic algorithms , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[19]  Aranyak Mehta,et al.  Progress in approximate nash equilibria , 2007, EC '07.

[20]  Karl Tuyls,et al.  Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent , 2019, IJCAI.

[21]  David Silver,et al.  A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning , 2017, NIPS.

[22]  S. Hart,et al.  A simple adaptive procedure leading to correlated equilibrium , 2000 .

[23]  Csaba Szepesvári,et al.  Bounds and dynamics for empirical game theoretic analysis , 2019, Autonomous Agents and Multi-Agent Systems.

[24]  John P. Dickerson,et al.  Using Inverse Optimization to Learn Cost Functions in Generalized Nash Games , 2022, Computers & Operations Research.

[25]  Yaodong Yang,et al.  Modelling Behavioural Diversity for Learning in Open-Ended Games , 2021, ICML.

[26]  John Fearnley,et al.  An Empirical Study of Finding Approximate Equilibria in Bimatrix Games , 2015, SEA.

[27]  Yaodong Yang,et al.  An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective , 2020, ArXiv.

[28]  Avrim Blum,et al.  Planning in the Presence of Cost Functions Controlled by an Adversary , 2003, ICML.

[29]  Nikos Vlassis,et al.  A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence I Mobk077-fm Synthesis Lectures on Artificial Intelligence and Machine Learning a Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence a Concise Introduction to Multiagent Systems and D , 2007 .

[30]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[31]  J. Zico Kolter,et al.  Large Scale Learning of Agent Rationality in Two-Player Zero-Sum Games , 2019, AAAI.

[32]  Eli Upfal,et al.  Learning Equilibria of Simulation-Based Games , 2019, ArXiv.

[33]  Adrian Vetta,et al.  Nash equilibria in competitive societies, with applications to facility location, traffic routing and auctions , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[34]  Paul G. Spirakis,et al.  Efficient Algorithms for Constant Well Supported Approximate Equilibria in Bimatrix Games , 2007, ICALP.

[35]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[36]  Wotao Yin,et al.  Learn to Predict Equilibria via Fixed Point Networks , 2021, ArXiv.

[37]  Paul G. Spirakis,et al.  An Optimization Approach for Approximate Nash Equilibria , 2007, WINE.

[38]  Kostas Margellos,et al.  Probably Approximately Correct Nash Equilibrium Learning , 2019, IEEE Transactions on Automatic Control.

[39]  R. Myerson.  Nash Equilibrium and the History of Economic Theory , 1999 .