Training Deep Convolutional Neural Networks to Play Go

Mastering the game of Go has remained a longstanding challenge to the field of AI. Modern computer Go programs rely on processing millions of possible future positions to play well, but intuitively a stronger and more 'humanlike' way to play the game would be to rely on pattern recognition rather than brute force computation. Following this sentiment, we train deep convolutional neural networks to play Go by training them to predict the moves made by expert Go players. To solve this problem we introduce a number of novel techniques, including a method of tying weights in the network to 'hard code' symmetries that are expected to exist in the target function, and demonstrate in an ablation study they considerably improve performance. Our final networks are able to achieve move prediction accuracies of 41.1% and 44.4% on two different Go datasets, surpassing previous state of the art on this task by significant margins. Additionally, while previous move prediction systems have not yielded strong Go playing programs, we show that the networks trained in this work acquired high levels of skill. Our convolutional neural networks can consistently defeat the well known Go program GNU Go and win some games against state of the art Go playing program Fuego while using a fraction of the play time.

[1]  Richard Bozulich,et al.  The Go player's almanac , 1992 .

[2]  Martin Müller,et al.  Computer Go , 2002, Artif. Intell..

[3]  Eric O. Postma,et al.  Local Move Prediction in Go , 2002, Computers and Games.

[4]  Thore Graepel,et al.  Bayesian pattern ranking for move prediction in the game of Go , 2006, ICML.

[5]  Rémi Coulom,et al.  Computing "Elo Ratings" of Move Patterns in the Game of Go , 2007, J. Int. Comput. Games Assoc..

[6]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[7]  Jun'ichi Tsujii,et al.  Move Prediction in Go with the Maximum Entropy Method , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[8]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[9]  Ilya Sutskever,et al.  Mimicking Go Experts with Convolutional Neural Networks , 2008, ICANN.

[10]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[11]  Martin Müller,et al.  Fuego—An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[12]  Shang-Rong Tsai,et al.  Current Frontiers in Computer Go , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[13]  Petr Baudis,et al.  PACHI: State of the Art Open Source Go Program , 2011, ACG.

[14]  Razvan Pascanu,et al.  Deep Learners Benefit More from Out-of-Distribution Examples , 2011, AISTATS.

[15]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[16]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[17]  Marco Platzner,et al.  Comparison of Bayesian move prediction systems for Computer Go , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[18]  Lars Schmidt-Thieme,et al.  Move Prediction in Go - Modelling Feature Interactions Using Latent Factors , 2013, KI.

[19]  David Silver,et al.  Move Evaluation in Go Using Deep Convolutional Neural Networks , 2014, ICLR.