What's the Next Move? Learning Player Strategies in Zoom Poker Games

In this article, we address the problem of modeling the actions of a human player in order to learn his strategies from his past game logs in Zoom Texas Hold'em poker variant. Although Texas Hold'em is a very popular game, Zoom is yet a very recent format of game in which, instead of playing in a specific table against a specific set of opponents, a player is placed in a large pool of players in which their opponents change every hand. Pros and cons of Zoom include respectively bigger effective time playing (and possibly getting money) and scarcity of data to get reads from the opponents. To deal with this problem, our model consists of a simple and generic set of features designed to fulfill each one of four proposed categories (hand quality, position insights, aggressiveness and current situation) in order to be able to capture a wide range of player strategies in each stage of the game. As a consequence of our modeling, we generate five data sets which were further evaluated by machine learning techniques. The results show that much of the player strategies were effectively learned, especially by non-linear techniques. Moreover, our data sets are available online as a test-bed for machine learning research in poker games.

[1]  David Sklansky,et al.  Hold'Em Poker for Advanced Players , 1999 .

[2]  Michael H. Bowling,et al.  Data Biased Robust Counter Strategies , 2009, AISTATS.

[3]  Lars Niklasson,et al.  Explaining Winning Poker--A Data Mining Approach , 2006, 2006 5th International Conference on Machine Learning and Applications (ICMLA'06).

[4]  Jonathan Schaeffer,et al.  Improved Opponent Modeling in Poker , 2000 .

[5]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[6]  David Sklansky,et al.  The Theory of Poker , 1999 .

[7]  Risto Miikkulainen,et al.  Evolving Adaptive Poker Players for Effective Opponent Exploitation , 2017, AAAI Workshops.

[8]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[9]  Kevin Waugh,et al.  DeepStack: Expert-level artificial intelligence in heads-up no-limit poker , 2017, Science.

[10]  Ian D. Watson,et al.  Case-based strategies in computer poker , 2012, AI Commun..

[11]  Jonathan Schaeffer,et al.  Opponent Modeling in Poker , 1998, AAAI/IAAI.

[12]  Luís Paulo Reis,et al.  A Profitable Online No-Limit Poker Playing Agent , 2014, 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT).

[13]  Duane Szafron,et al.  Using counterfactual regret minimization to create competitive multiplayer poker agents , 2010, AAMAS 2010.

[14]  Tuomas Sandholm,et al.  Hierarchical Abstraction, Distributed Equilibrium Computation, and Post-Processing, with Application to a Champion No-Limit Texas Hold'em Agent , 2015, AAAI Workshop: Computer Poker and Imperfect Information.

[15]  Guy Van den Broeck,et al.  Monte-Carlo Tree Search in Poker Using Expected Reward Distributions , 2009, ACML.

[16]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[17]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[18]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.