Game-Tree Search with Adaptation in Stochastic Imperfect-Information Games

Building a high-performance poker-playing program is a challenging project. The best program to date, PsOpti, uses game theory to solve a simplified version of the game. Although the program plays reasonably well, it is oblivious to the opponent's weaknesses and biases. Modeling the opponent to exploit predictability is critical to success at poker. This paper introduces Vexbot, a program that uses a game-tree search algorithm to compute the expected value of each betting option, and does real-time opponent modeling to improve its evaluation function estimates. The result is a program that defeats PsOpti convincingly, and poses a much tougher challenge for strong human players.

[1]  Eric Horvitz,et al.  The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users , 1998, UAI.

[2]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[3]  Fredrik A. Dahl,et al.  A Reinforcement Learning Algorithm Applied to Simplified Two-Player Texas Hold'em Poker , 2001, ECML.

[4]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[5]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[6]  J. Neumann,et al.  The Theory of Games and Economic Behaviour , 1944 .

[7]  Darse Billings,et al.  The First International RoShamBo Programming Competition , 2000, J. Int. Comput. Games Assoc..

[8]  Jonathan Schaeffer,et al.  Opponent Modeling in Poker , 1998, AAAI/IAAI.

[9]  H. Kuhn.  9. A SIMPLIFIED TWO-PERSON POKER , 1951 .

[10]  Peter Brusilovsky,et al.  User Modeling 2003 , 2003, Lecture Notes in Computer Science.

[11]  David Carmel,et al.  Incorporating Opponent Models into Adversary Search , 1996, AAAI/IAAI, Vol. 1.

[12]  Kevin B. Korb,et al.  Bayesian Poker , 1999, UAI.

[13]  Nicholas V. Findler,et al.  Studies in machine cognition using the game of poker , 1977, CACM.

[14]  David Carmel,et al.  Learning and using opponent models in adversary search , 1996 .

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  Hiroyuki Iida,et al.  Potential Applications of Opponent-Model Search , 1994, J. Int. Comput. Games Assoc..

[17]  Krzysztof Z. Gajos,et al.  Automatically personalizing user interfaces , 2003, IJCAI 2003.

[18]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[19]  Jonathan Schaeffer,et al.  Using Probabilistic Knowledge and Simulation to Play Poker , 1999, AAAI/IAAI.

[20]  Peter J. Jansen,et al.  Using knowledge about the opponent in game-tree search , 1992 .

[21]  Michael Buro,et al.  Solving the Oshi-Zumo Game , 2003, ACG.

[22]  Darse Billings.  VEXBOT WINS POKER TOURNAMENT , 2003 .