Run the GAMUT: a comprehensive approach to evaluating game-theoretic algorithms

We present GAMUT^1, a suite of game generators designed for testing game-theoretic algorithms. We explain why such a generator is necessary, offer a way of visualizing relationships between the sets of games supported by GAMUT, and give an overview of GAMUTýs architecture. We highlight the importance of using comprehensive test data by benchmarking existing algorithms. We show surprisingly large variation in algorithm performance across different sets of games for two widely-studied problems: computing Nash equilibria and multiagent learning in repeated games.

[1]  Christos H. Papadimitriou,et al.  Algorithms, games, and the internet , 2001, STOC '01.

[2]  Vincent Conitzer,et al.  Complexity Results about Nash Equilibria , 2002, IJCAI.

[3]  Robert Wilson,et al.  A global Newton method to compute Nash equilibria , 2003, J. Econ. Theory.

[4]  Anatol Rapoport,et al.  The 2x2 Game , 1976 .

[5]  R. Rosenthal.  A class of games possessing pure-strategy Nash equilibria , 1973 .

[6]  R. McKelvey,et al.  Computation of equilibria in finite games , 1996 .

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  Daphne Koller,et al.  A Continuation Method for Nash Equilibria in Structured Games , 2003, IJCAI.

[9]  William H. Ruckle.  Geometric games and their applications , 1983 .

[10]  J. Mertens,et al.  ON THE STRATEGIC STABILITY OF EQUILIBRIA , 1986 .

[11]  Eitan Zemel,et al.  Nash and correlated equilibria: Some complexity considerations , 1989 .

[12]  B. Stengel,et al.  COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES , 1996 .

[13]  C. E. Lemke,et al.  Equilibrium Points of Bimatrix Games , 1964 .

[14]  A. Talman,et al.  Simplicial variable dimension algorithms for solving the nonlinear complementarity problem on a product of unit simplices using a general labelling , 1987 .

[15]  P. Gács,et al.  Algorithms , 1992 .

[16]  Christos H. Papadimitriou,et al.  Algorithms, Games, and the Internet , 2001, ICALP.

[17]  Manuela M. Veloso,et al.  Rational and Convergent Learning in Stochastic Games , 2001, IJCAI.

[18]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[19]  Marco Scarsini,et al.  On the Number of Pure Strategy Nash Equilibria in Random Games , 2000, Games Econ. Behav..

[20]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[21]  Daphne Koller,et al.  Multi-agent algorithms for solving graphical games , 2002, AAAI/IAAI.

[22]  Yoav Shoham,et al.  Learning the Empirical Hardness of Optimization Problems: The Case of Combinatorial Auctions , 2002, CP.

[23]  Paul R. Milgrom,et al.  Rationalizability, Learning, and Equilibrium in Games with Strategic Complementarities , 1990 .