Nested Monte Carlo Search for Two-Player Games

The use of the Monte Carlo playouts as an evaluation function has proved to be a viable, general technique for searching intractable game spaces. This facilitate the use of statistical techniques like Monte Carlo Tree Search (MCTS), but is also known to require significant processing overhead. We seek to improve the quality of information extracted from the Monte Carlo playout in three ways. Firstly, by nesting the evaluation function inside another evaluation function; secondly, by measuring and utilising the depth of the playout; and thirdly, by incorporating pruning strategies that eliminate unnecessary searches and avoid traps. Our experimental data, obtained on a variety of two-player games from past General Game Playing (GGP) competitions and others, demonstrate the usefulness of these techniques in a Nested Player when pitted against a standard, optimised UCT player.

[1]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[2]  Timo Ewalds.  Playing and Solving Havannah , 2012 .

[3]  Nicolas Jouandeau,et al.  Solving breakthrough with Race Patterns and Job-Level Proof Number Search , 2011, ACG.

[4]  Christopher D. Rosin,et al.  Nested Rollout Policy Adaptation for Monte Carlo Tree Search , 2011, IJCAI.

[5]  Michael Thielscher,et al.  General Game Playing , 2015 .

[6]  Tristan Cazenave,et al.  Score Bounded Monte-Carlo Tree Search , 2010, Computers and Games.

[7]  Mark H. M. Winands,et al.  αβ-based play-outs in Monte-Carlo Tree Search , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[8]  Donald E. Knuth,et al.  The Solution for the Branching Factor of the Alpha-Beta Pruning Algorithm , 1981, ICALP.

[9]  Mark H. M. Winands,et al.  Monte-Carlo Tree Search and minimax hybrids , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[10]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[11]  Jean Méhat,et al.  Combining UCT and Nested Monte Carlo Search for Single-Player General Game Playing , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[12]  Gerald Tesauro,et al.  Monte-Carlo simulation balancing , 2009, ICML '09.

[13]  Olivier Teytaud,et al.  Meta Monte-Carlo Tree Search for Automatic Opening Book Generation , 2009 .

[14]  Mark H. M. Winands,et al.  Quality-based Rewards for Monte-Carlo Tree Search Simulations , 2014, ECAI.

[15]  Stefan Freyr Gudmundsson,et al.  Sufficiency-Based Selection Strategy for MCTS , 2013, IJCAI.

[16]  Michael Buro,et al.  Recursive Monte Carlo search for imperfect information games , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[17]  Kanako Komiya,et al.  Nested Monte-Carlo Search with simulation reduction , 2012, Knowl. Based Syst..

[18]  Michael R. Genesereth,et al.  General Game Playing: Overview of the AAAI Competition , 2005, AI Mag..

[19]  Richard J. Lorentz.  Improving Monte-Carlo Tree Search in Havannah , 2010, Computers and Games.

[20]  Yngvi Björnsson,et al.  CadiaPlayer: A Simulation-Based General Game Player , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[21]  Dana S. Nau,et al.  An Analysis of Forward Pruning , 1994, AAAI.

[22]  Tristan Cazenave,et al.  Nested Monte-Carlo Expression Discovery , 2010, ECAI.

[23]  Tristan Cazenave,et al.  Nested Monte-Carlo Search , 2009, IJCAI.