Opponent Modeling in Poker Games

Texas Hold’em poker is a popular game worldwide and it attracts increasing attention from community of artificial intelligence as a typical decision-making problem in non-deterministic and incomplete information environment. One of the key tasks to deal with the poker game is the opponent modeling, which aims to exploit the opponent weakness based on history behaviors. In this paper, we study mixed-method opponent modeling, one is Bayesian probabilistic model, one is the neural network (NN)-based prediction model, and the last is opponent type identifying model. Then, we combine these three methods to generate an integrated agent for opponent modeling. The opponents are categorized into 4 types according to their risk preference of strategies. The main step in Bayesian method is calculating the posterior distribution over opponent’s strategy space and selecting the maximum probability. The main idea of NN-based method is using observation data to improve the prediction accuracy of opponent’s hand. The main idea of opponent type identifying model is building a classifier with two factors. Finally, we design a simplified poker game to conduct experiment and demonstrate the effectiveness of our methods.

[1]  Risto Miikkulainen,et al.  Dynamic Adaptation and Opponent Exploitation in Computer Poker , 2018, AAAI Workshops.

[2]  Jonathan Schaeffer,et al.  Improved Opponent Modeling in Poker , 2000 .

[3]  Michael H. Bowling,et al.  Bayes' Bluff: Opponent Modelling in Poker , 2005, UAI 2005.

[4]  Jonathan Schaeffer,et al.  Opponent Modeling in Poker , 1998, AAAI/IAAI.

[5]  Gautam Bhat.  NN-based Poker Hand Classification and Game Playing , 2016 .

[6]  Tuomas Sandholm,et al.  Game theory-based opponent modeling in large imperfect-information games , 2011, AAMAS.

[7]  Luís Paulo Reis,et al.  Adapting Strategies to Opponent Models in Incomplete Information Games: A Reinforcement Learning Approach for Poker , 2012, AIS.

[8]  International Foundation for Autonomous Agents and MultiAgent Systems ( IFAAMAS ) , 2007 .

[9]  Noam Brown,et al.  Superhuman AI for multiplayer poker , 2019, Science.

[10]  Stephen J. Roberts,et al.  Learning Against Non-Stationary Agents with Opponent Modelling and Deep Reinforcement Learning , 2018, AAAI Spring Symposia.

[11]  Risto Miikkulainen,et al.  Evolving Adaptive LSTM Poker Players for Effective Opponent Exploitation , 2016 .

[12]  Kevin Waugh,et al.  Abstraction pathologies in extensive games , 2009, AAMAS.

[13]  Kevin Waugh,et al.  DeepStack: Expert-level artificial intelligence in heads-up no-limit poker , 2017, Science.

[14]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[15]  David Silver,et al.  Deep Reinforcement Learning from Self-Play in Imperfect-Information Games , 2016, ArXiv.

[16]  Neil Burch,et al.  Heads-up limit hold’em poker is solved , 2015, Science.

[17]  Bret Hoehn,et al.  Effective short-term opponent exploitation in simplified poker , 2005, Machine Learning.