Finding equilibria in large sequential games of imperfect information

Finding an equilibrium of an extensive form game of imperfect information is a fundamental problem in computational game theory, but current techniques do not scale to large games. To address this, we introduce the ordered game isomorphism and the related ordered game isomorphic abstraction transformation. For a multi-player sequential game of imperfect information with observable actions and an ordered signal space, we prove that any Nash equilibrium in an abstracted smaller game, obtained by one or more applications of the transformation, can be easily converted into a Nash equilibrium in the original game. We present an algorithm, GameShrink, for abstracting the game using our isomorphism exhaustively. Its complexity is Õ(n2), where n is the number of nodes in a structure we call the signal tree. It is no larger than the game tree, and on nontrivial games it is drastically smaller, so GameShrink has time and space complexity sublinear in the size of the game tree. Using GameShrink, we find an equilibrium to a poker game with 3.1 billion nodes--over four orders of magnitude more than in the largest poker game solved previously. We discuss several electronic commerce applications for GameShrink. To address even larger games, we introduce approximation methods that do not preserve equilibrium, but nevertheless yield (ex post) provably close-to-optimal strategies.

[1]  Bernhard von Stengel,et al.  Exponentially many steps for finding a Nash equilibrium in a bimatrix game , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[2]  D. R. Fulkerson,et al.  Flows in Networks. , 1964 .

[3]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[4]  H. Kuhn 9. A SIMPLIFIED TWO-PERSON POKER , 1951 .

[5]  Matthew L. Ginsberg,et al.  Partition Search , 1996, AAAI/IAAI, Vol. 1.

[6]  Tuomas Sandholm,et al.  A Texas Hold'em poker player based on automated abstraction and real-time equilibrium computation , 2006, AAMAS '06.

[7]  Moshe Tennenholtz,et al.  Local-Effect Games , 2003, IJCAI.

[8]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[9]  Dana S. Nau,et al.  Computer Bridge - A Big Win for AI Planning , 1998, AI Mag..

[10]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .

[11]  Christos H. Papadimitriou,et al.  Algorithms, games, and the internet , 2001, STOC '01.

[12]  R BELLMAN,et al.  Some two person games involving bluffing. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Matthew L. Ginsberg,et al.  GIB: Steps Toward an Expert-Level Bridge-Playing Program , 1999, IJCAI.

[14]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[15]  P. Reny,et al.  On the Strategic Equivalence of Extensive Form Games , 1994 .

[16]  D. Koller,et al.  The complexity of two-person zero-sum games in extensive form , 1992 .

[17]  D. Koller,et al.  Finding mixed strategies with small supports in extensive form games , 1996 .

[18]  Ilya Segal,et al.  Solutions manual for Microeconomic theory : Mas-Colell, Whinston and Green , 1997 .

[19]  Michael P. Wellman,et al.  On state-space abstraction for anytime evaluation of Bayesian networks , 1996, SGAR.

[20]  André Casajus,et al.  Weak isomorphism of extensive games , 2003, Math. Soc. Sci..

[21]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[22]  C. E. Lemke,et al.  Equilibrium Points of Bimatrix Games , 1964 .

[23]  J. M. Bilbao,et al.  Contributions to the Theory of Games , 2005 .

[24]  Kevin Leyton-Brown,et al.  Computing Nash Equilibria of Action-Graph Games , 2004, UAI.

[25]  Bernhard von Stengel,et al.  Fast algorithms for finding randomized strategies in game trees , 1994, STOC '94.

[26]  J. Nash Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[27]  B. Stengel,et al.  Efficient Computation of Behavior Strategies , 1996 .

[28]  Tuomas Sandholm,et al.  Optimal Rhode Island Hold'em Poker , 2005, AAAI.

[29]  David M. Kreps,et al.  Sequential Equilibria Author ( s ) : , 1982 .

[30]  Yuval Rabani,et al.  Linear Programming , 2007, Handbook of Approximation Algorithms and Metaheuristics.

[31]  Xiaotie Deng,et al.  Settling the Complexity of Two-Player Nash Equilibrium , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[32]  Peter Bro Miltersen,et al.  Computing sequential equilibria for two-player games , 2006, SODA '06.

[33]  Robert E. Tarjan,et al.  Efficiency of a Good But Not Linear Set Union Algorithm , 1972, JACM.

[34]  Michael L. Littman,et al.  Abstraction Methods for Game Theoretic Poker , 2000, Computers and Games.

[35]  Xiaotie Deng,et al.  On the complexity of equilibria , 2002, STOC '02.

[36]  Amin Saberi,et al.  Approximating Market Equilibria , 2003, RANDOM-APPROX.

[37]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[38]  Tuomas Sandholm,et al.  A Competitive Texas Hold'em Poker Player via Automated Abstraction and Real-Time Equilibrium Computation , 2006, AAAI.

[39]  Andrés Perea,et al.  Rationality in extensive form games , 2001 .

[40]  Reinhard Selten,et al.  Evolutionary stability in extensive two-person games - correction and further development , 1988 .

[41]  Daphne Koller,et al.  A Continuation Method for Nash Equilibria in Structured Games , 2003, IJCAI.

[42]  Stephen J. Wright Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[43]  Yoav Shoham,et al.  Simple search methods for finding a Nash equilibrium , 2004, Games Econ. Behav..

[44]  B. Stengel,et al.  COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES , 1996 .

[45]  Vincent Conitzer,et al.  Mixed-Integer Programming Methods for Finding Nash Equilibria , 2005, AAAI.

[46]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[47]  Tim Roughgarden,et al.  Computing equilibria in multi-player games , 2005, SODA '05.

[48]  Nikhil R. Devanur,et al.  Market equilibrium via a primal-dual-type algorithm , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[49]  Herbert E. Scarf,et al.  The Approximation of Fixed Points of a Continuous Mapping , 1967 .

[50]  Daniel Dominic Sleator,et al.  Computer analysis of Sprouts , 1999 .

[51]  J. Mertens,et al.  ON THE STRATEGIC STABILITY OF EQUILIBRIA , 1986 .

[52]  Tuomas Sandholm,et al.  Sequences of take-it-or-leave-it offers: near-optimal auctions without full valuation revelation , 2003, AAMAS '06.

[53]  L. S. Shapley,et al.  10. A SIMPLE THREE-PERSON POKER GAME , 1951 .

[54]  Robert Wilson Computing Equilibria of Two-Person Games from the Extensive Form , 1972 .

[55]  Aranyak Mehta,et al.  Playing large games using simple strategies , 2003, EC '03.

[56]  Robert Wilson,et al.  A global Newton method to compute Nash equilibria , 2003, J. Econ. Theory.

[57]  A. Mas-Colell,et al.  Microeconomic Theory , 1995 .

[58]  D. Koller,et al.  Efficient Computation of Equilibria for Extensive Two-Person Games , 1996 .

[59]  Craig A. Knoblock Automatically Generating Abstractions for Planning , 1994, Artif. Intell..

[60]  R. McKelvey,et al.  Computation of equilibria in finite games , 1996 .

[61]  H. W. Kuhn,et al.  11. Extensive Games and the Problem of Information , 1953 .

[62]  W. Ackermann Zum Hilbertschen Aufbau der reellen Zahlen , 1928 .