Active Opening Book Application for Monte-Carlo Tree Search in 19×19 Go

The dominant approach for programs playing the Asian board game of Go is nowadays Monte-Carlo Tree Search (MCTS). However, MCTS does not perform well in the opening phase of the game, as the branching factor is high and consequences of moves can be far delayed. Human knowledge about Go openings is typically captured in joseki, local sequences of moves that are considered optimal for both players. The choice of the correct joseki in a given whole-board position, however, is difficult to formalize. This paper presents an approach to successfully apply global as well as local opening moves, extracted from databases of high-level game records, in the MCTS framework. Instead of blindly playing moves that match local joseki patterns (passive opening book application), knowledge about these moves is integrated into the search algorithm by the techniques of move pruning and move biasing (active opening book application). Thus, the opening book serves to nudge the search into the direction of tried and tested local moves, while the search is able to filter out locally optimal, but globally problematic move choices. In our experiments, active book application outperforms passive book application and plain MCTS in 19×19 Go.

[1]  Peter Drake,et al.  Using Human knowledge to Improve Opening Strategy in Computer Go , 2010, IC-AI.

[2]  Shang-Rong Tsai,et al.  Current Frontiers in Computer Go , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[3]  Robert M. Hyatt,et al.  Book Learning - a Methodology to Tune an Opening Book Automatically , 1999, J. Int. Comput. Games Assoc..

[4]  Thomas R. Lincke,et al.  Strategies for the Automatic Construction of Opening Books , 2000, Computers and Games.

[5]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[6]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[7]  Olivier Teytaud,et al.  Meta Monte-Carlo Tree Search for Automatic Opening Book Generation , 2009 .

[8]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[9]  Hendrik Baier,et al.  The Power of Forgetting: Improving the Last-Good-Reply Policy in Monte Carlo Go , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[10]  Rémi Coulom,et al.  Computing "Elo Ratings" of Move Patterns in the Game of Go , 2007, J. Int. Comput. Games Assoc..

[11]  Tzung-Pei Hong,et al.  The Computational Intelligence of MoGo Revealed in Taiwan's Computer Go Tournaments , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[12]  Donald E. Eastlake,et al.  The Greenblatt chess program , 1967, AFIPS '67 (Fall).

[13]  Richard Lorentz,et al.  Generating an Opening Book for Amazons , 2004, Computers and Games.

[14]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[15]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[16]  Tai-Ning Yang,et al.  M-eval: A Multivariate Evaluation Function for Opening Positions in Computer Go , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[17]  Michael Buro Toward Opening Book Learning , 1999, J. Int. Comput. Games Assoc..

[18]  Ulf Lorenz,et al.  Innovative Opening-Book Handling , 2006, ACG.

[19]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[20]  Jonathan Schaeffer,et al.  CHINOOK: The World Man-Machine Checkers Champion , 1996, AI Mag..

[21]  Donald C. Wunsch,et al.  Computer Go: A Grand Challenge to AI , 2007, Challenges for Computational Intelligence.

[22]  David Silver,et al.  Combining online and offline knowledge in UCT , 2007, ICML '07.