Case-based strategies in computer poker

The state-of-the-art within Artificial Intelligence has directly benefited from research conducted within the computer poker domain. One such success has been the advancement of bottom up equilibrium finding algorithms via computational game theory. On the other hand, alternative top down approaches, that attempt to generalise decisions observed within a collection of data, have not received as much attention. In this work we employ a top down approach in order to construct case-based strategies within three computer poker domains. Our analysis begins within the simplest variation of Texas Hold'em poker, i.e. two-player, limit Hold'em. We trace the evolution of our case-based architecture and evaluate the effect that modifications have on strategy performance. The end result of our experimentation is a coherent framework for producing strong case-based strategies based on the observation and generalisation of expert decisions. The lessons learned within this domain offer valuable insights, that we use to apply the framework to the more complicated domains of two-player, no-limit Hold'em and multi-player, limit Hold'em. For each domain we present results obtained from the Annual Computer Poker Competition, where the best poker agents in the world are challenged against each other. We also present results against human opposition.

[1]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[2]  Santiago Ontañón,et al.  On-Line Case-Based Plan Adaptation for Real-Time Strategy Games , 2008, AAAI.

[3]  Ian D. Watson,et al.  Similarity-Based Retrieval and Solution Re-use Policies in the Game of Texas Hold'em , 2010, ICCBR.

[4]  David Sklansky,et al.  Hold'Em Poker for Advanced Players , 1999 .

[5]  Babak Esfandiari,et al.  A Case-Based Reasoning Approach to Imitating RoboCup Players , 2008, FLAIRS.

[6]  Guy Van den Broeck,et al.  Monte-Carlo Tree Search in Poker Using Expected Reward Distributions , 2009, ACML.

[7]  Jay H. Powell,et al.  Utilizing Case-Based Reasoning and Automatic Case Elicitation to Develop a Self-Taught Knowledgeable Agent , 2004 .

[8]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1988, IJCAI 1989.

[9]  Kevin Waugh,et al.  Abstraction pathologies in extensive games , 2009, AAMAS.

[10]  Ian D. Watson,et al.  On Combining Decisions from Multiple Expert Imitators for Performance , 2011, IJCAI.

[11]  Rickard Andersson.  Pseudo-Optimal Strategies in No-Limit Poker , 2006, J. Int. Comput. Games Assoc..

[12]  Santiago Ontañón,et al.  ON‐LINE CASE‐BASED PLANNING , 2010, Comput. Intell..

[13]  Michael H. Bowling,et al.  Computing Robust Counter-Strategies , 2007, NIPS.

[14]  Ian D. Watson,et al.  Applying case-based reasoning - techniques for the enterprise systems , 1997 .

[15]  Martin Zinkevich,et al.  The Annual Computer Poker Competition , 2013, AI Mag..

[16]  Javier Peña,et al.  Gradient-Based Algorithms for Finding Nash Equilibria in Extensive Form Games , 2007, WINE.

[17]  Duane Szafron,et al.  Using counterfactual regret minimization to create competitive multiplayer poker agents , 2010, AAMAS 2010.

[18]  Christopher K. Riesbeck,et al.  Inside Case-Based Reasoning , 1989 .

[19]  Santiago Ontañón,et al.  Case-Based Planning and Execution for Real-Time Strategy Games , 2007, ICCBR.

[20]  Ashwin Ram,et al.  Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL , 2007, IJCAI.

[21]  Santiago Ontañón,et al.  Situation Assessment for Plan Retrieval in Real-Time Strategy Games , 2008, ECCBR.

[22]  Hector Muñoz-Avila,et al.  Recognizing the Enemy: Combining Reinforcement Learning with Strategy Selection Using Case-Based Reasoning , 2008, ECCBR.

[23]  Katia Sycara,et al.  CADET: a case-based synthesis tool for engineering design , 1991 .

[24]  David Sklansky,et al.  The Theory of Poker , 1999 .

[25]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[26]  Troels Bjerre Lund,et al.  A heads-up no-limit Texas Hold'em poker player: discretized betting models and automatically generated equilibrium-finding programs , 2008, AAMAS.

[27]  David W. Aha,et al.  Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game , 2005, Künstliche Intell..

[28]  Jonathan Schaeffer,et al.  Game-Tree Search with Adaptation in Stochastic Imperfect-Information Games , 2004, Computers and Games.

[29]  Kristian J. Hammond,et al.  Case-based planning: A framework for planning from experience ☆ , 1990 .

[30]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[31]  Tuomas Sandholm,et al.  Better automated abstraction techniques for imperfect information games, with application to Texas Hold'em poker , 2007, AAMAS '07.