Information Set Monte Carlo Tree Search

Monte Carlo tree search (MCTS) is an AI technique that has been successfully applied to many deterministic games of perfect information. This paper investigates the application of MCTS methods to games with hidden information and uncertainty. In particular, three new information set MCTS (ISMCTS) algorithms are presented which handle different sources of hidden information and uncertainty in games. Instead of searching minimax trees of game states, the ISMCTS algorithms search trees of information sets, more directly analyzing the true structure of the game. These algorithms are tested in three domains with different characteristics, and it is demonstrated that our new algorithms outperform existing approaches to handling hidden information and uncertainty in games.

[1]  H. Jaap van den Herik,et al.  The Advantage of the Initiative , 2000, Inf. Sci..

[2]  Olivier Teytaud,et al.  Upper Confidence Trees with Short Term Partial Information , 2011, EvoApplications.

[3]  Peter I. Cowling,et al.  Determinization and information set Monte Carlo Tree Search for the card game Dou Di Zhu , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[4]  Guillaume Chaslot,et al.  Integrating Opponent Models with Monte-Carlo Tree Search in Poker , 2010, Interactive Decision Theory and Game Theory.

[5]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[6]  H. Jaap van den Herik,et al.  GO‐MOKU SOLVED BY NEW SEARCH TECHNIQUES , 1996, Comput. Intell..

[7]  Ian Frank,et al.  Search in Games with Incomplete Information: A Case Study Using Bridge Card Play , 1998, Artif. Intell..

[8]  Olivier Teytaud,et al.  Lemmas on partial observation, with application to phantom games , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[9]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[10]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[11]  Levente Kocsis,et al.  Transpositions and move groups in Monte Carlo tree search , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[12]  Guillaume Chaslot,et al.  A Comparison of Monte-Carlo Methods for Phantom Go , 2007 .

[13]  H. Jaap van den Herik,et al.  Games solved: Now and in the future , 2002, Artif. Intell..

[14]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[15]  E. S. Pearson,et al.  THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL , 1934 .

[16]  Nicolò Cesa-Bianchi,et al.  Gambling in a rigged casino: The adversarial multi-armed bandit problem , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[17]  Ian Frank,et al.  Search in Games with Incomplete Information: A Case Study Using Bridge Card Play , 1998, Artificial Intelligence.

[18]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[19]  David Auger,et al.  Multiple Tree for Partially Observable Monte-Carlo Tree Search , 2011, EvoApplications.

[20]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[21]  Paolo Ciancarini,et al.  Monte Carlo tree search in Kriegspiel , 2010, Artif. Intell..

[22]  P. Cowling,et al.  Determinization in Monte-Carlo Tree Search for the card game , 2011 .

[23]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[24]  Alan Fern,et al.  Ensemble Monte-Carlo Planning: An Empirical Study , 2011, ICAPS.

[25]  J. Schaeffer,et al.  Comparing UCT versus CFR in Simultaneous Games , 2009 .

[26]  Mark Richards,et al.  Opponent Modeling in Scrabble , 2007, IJCAI.

[27]  Alan Fern,et al.  Lower Bounding Klondike Solitaire with Monte-Carlo Planning , 2009, ICAPS.

[28]  Matthew L. Ginsberg,et al.  GIB: Imperfect Information in a Computationally Challenging Game , 2011, J. Artif. Intell. Res..

[29]  Kevin Waugh,et al.  Monte Carlo Sampling for Regret Minimization in Extensive Games , 2009, NIPS.

[30]  Nathan R. Sturtevant,et al.  Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search , 2010, AAAI.

[31]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[32]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[33]  Mark H. M. Winands,et al.  Monte-Carlo Tree Search for the game of Scotland Yard , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[34]  Robert Givan,et al.  A framework for simulation-based network control via hindsight optimization , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[35]  David Silver,et al.  Monte-Carlo tree search and rapid action value estimation in computer Go , 2011, Artif. Intell..

[36]  Marc Lanctot,et al.  Computing Approximate Nash Equilibria and Robust Best-Responses Using Sampling , 2011, J. Artif. Intell. Res..