Position-based reinforcement learning biased MCTS for General Video Game Playing

This paper proposes an application of reinforcement learning and position-based features in rollout bias training of Monte-Carlo Tree Search (MCTS) for General Video Game Playing (GVGP). As an improvement on Knowledge-based Fast-Evo MCTS proposed by Perez et al., the proposed method is designated for both the GVG-AI Competition and improvement of the learning mechanism of the original method. The performance of the proposed method is evaluated empirically, using all games from six training sets available in the GVG-AI Framework, and the proposed method achieves better scores than five other existing MCTS-based methods overall.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[3]  Martin A. Riedmiller,et al.  Batch Reinforcement Learning , 2012, Reinforcement Learning.

[4]  David Silver,et al.  Reinforcement learning and simulation-based search in computer go , 2009 .

[5]  Simon M. Lucas,et al.  Knowledge-based fast evolutionary MCTS for general video game playing , 2014, 2014 IEEE Conference on Computational Intelligence and Games.

[6]  Julian Togelius,et al.  General Video Game AI: Competition, Challenges and Opportunities , 2016, AAAI.

[7]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[8]  Marc G. Bellemare,et al.  Investigating Contingency Awareness Using Atari 2600 Games , 2012, AAAI.

[9]  Branko Ster,et al.  Enhancing upper confidence bounds for trees with temporal difference values , 2014, 2014 IEEE Conference on Computational Intelligence and Games.

[10]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[11]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[12]  Yavar Naddaf,et al.  Game-independent AI agents for playing Atari 2600 console games , 2010 .

[13]  Simon M. Lucas,et al.  Open Loop Search for General Video Game Playing , 2015, GECCO.

[14]  Julian Togelius,et al.  Investigating MCTS modifications in general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[15]  Ruck Thawonmas,et al.  Combining pathfmding algorithm with Knowledge-based Monte-Carlo tree search in general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[16]  Risto Miikkulainen,et al.  A Neuroevolution Approach to General Atari Game Playing , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[17]  Kyung-Joong Kim,et al.  MCTS with influence map for general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[18]  J. G. van Eeden.  Analysing And Improving The Knowledge-based Fast Evolutionary MCTS Algorithm , 2015 .