A Competitive Texas Hold'em Poker Player via Automated Abstraction and Real-Time Equilibrium Computation

We present a game theory-based heads-up Texas Hold'em poker player, GS1. To overcome the computational obstacles stemming from Texas Hold'em's gigantic game tree, the player employs our automated abstraction techniques to reduce the complexity of the strategy computations. Texas Hold'em consists of four betting rounds. Our player solves a large linear program (offline) to compute strategies for the abstracted first and second rounds. After the second betting round, our player updates the probability of each possible hand based on the observed betting actions in the first two rounds as well as the revealed cards. Using these updated probabilities, our player computes in real-time an equilibrium approximation for the last two abstracted rounds. We demonstrate that our player, which incorporates very little poker-specific knowledge, is competitive with leading poker-playing programs which incorporate extensive domain knowledge, as well as with advanced human players.

[1]  A. Odlyzko Proceedings of the ACM Conference on Electronic Commerce , 2003 .

[2]  Tuomas Sandholm,et al.  Finding equilibria in large sequential games of imperfect information , 2006, EC '06.

[3]  H W Kuhn Extensive Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Dana S. Nau,et al.  Computer Bridge - A Big Win for AI Planning , 1998, AI Mag..

[5]  Jonathan Schaeffer,et al.  Game-Tree Search with Adaptation in Stochastic Imperfect-Information Games , 2004, Computers and Games.

[6]  Michael P. Wellman,et al.  On state-space abstraction for anytime evaluation of Bayesian networks , 1996, SGAR.

[7]  Bret Hoehn,et al.  Effective short-term opponent exploitation in simplified poker , 2005, Machine Learning.

[8]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[9]  Michael L. Littman,et al.  Abstraction Methods for Game Theoretic Poker , 2000, Computers and Games.

[10]  R BELLMAN,et al.  Some two person games involving bluffing. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Matthew L. Ginsberg,et al.  GIB: Steps Toward an Expert-Level Bridge-Playing Program , 1999, IJCAI.

[12]  B. Stengel,et al.  Efficient Computation of Behavior Strategies , 1996 .

[13]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[14]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[15]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[16]  L. S. Shapley,et al.  10. A SIMPLE THREE-PERSON POKER GAME , 1951 .

[17]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[18]  H. Kuhn 9. A SIMPLIFIED TWO-PERSON POKER , 1951 .

[19]  Craig A. Knoblock Automatically Generating Abstractions for Planning , 1994, Artif. Intell..

[20]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[21]  Peter Bro Miltersen,et al.  Computing sequential equilibria for two-player games , 2006, SODA '06.

[22]  D. Koller,et al.  Efficient Computation of Equilibria for Extensive Two-Person Games , 1996 .

[23]  Michael H. Bowling,et al.  Bayes' Bluff: Opponent Modelling in Poker , 2005, UAI 2005.

[24]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[25]  Matthew L. Ginsberg,et al.  Partition Search , 1996, AAAI/IAAI, Vol. 1.