Modifying MCTS for Human-Like General Video Game Playing

We address the problem of making general video game playing agents play in a human-like manner. To this end, we introduce several modifications of the UCT formula used in Monte Carlo Tree Search that biases action selection towards repeating the current action, making pauses, and limiting rapid switching between actions. Playtraces of human players are used to model their propensity for repeated actions; this model is used for biasing the UCT formula. Experiments show that our modified MCTS agent, called BoT, plays quantitatively similar to human players as measured by the distribution of repeated actions. A survey of human observers reveals that the agent exhibits human-like playing style in some games but not others.

[1]  Christian Bauckhage,et al.  Learning Human-Like Movement Behavior for Computer Games , 2004 .

[2]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[3]  Julian Togelius,et al.  Imitating human playing styles in Super Mario Bros , 2013, Entertain. Comput..

[4]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[5]  Herbert A. Simon,et al.  The game of chess , 1992 .

[6]  Risto Miikkulainen,et al.  UT2: Human-like behavior via neuroevolution of combat behavior and replay of human traces , 2011, CIG.

[7]  Julian Togelius,et al.  Search-Based Procedural Content Generation: A Taxonomy and Survey , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[8]  Aaron Sloman,et al.  What Sort of Architecture is Required for a Human-Like Agent? , 1999 .

[9]  Philip Hingston,et al.  Believable Bots , 2012, Springer Berlin Heidelberg.

[10]  Mark J. Nelson.  Game Metrics Without Players: Strategies for Understanding Game Artifacts , 2011, Artificial Intelligence in the Game Design Process.

[11]  Julian Togelius,et al.  Monte Mario: platforming with MCTS , 2014, GECCO.

[12]  Andrew Nealen,et al.  Exploring Game Space Using Survival Analysis , 2015, FDG.

[13]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[14]  Michael R. Genesereth,et al.  General Game Playing: Overview of the AAAI Competition , 2005, AI Mag..

[15]  Rainer Malaka.  Entertainment Computing , 2014, Informatik-Spektrum.

[16]  Araceli Sanchis,et al.  A human-like TORCS controller for the Simulated Car Racing Championship , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[17]  Julian Togelius,et al.  Ieee Transactions on Computational Intelligence and Ai in Games the 2014 General Video Game Playing Competition , 2022 .

[18]  Philip Hingston.  The 2K BotPrize , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[19]  J. Meigs,et al.  WHO Technical Report , 1954, The Yale Journal of Biology and Medicine.

[20]  Julian Togelius,et al.  The turing test track of the 2012 Mario AI Championship: Entries and evaluation , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[21]  M. Mon-Williams,et al.  Motor Control and Learning , 2006 .

[22]  Randall D. Beer,et al.  Simulation of adaptive behavior , 1991, Current Opinion in Neurobiology.

[23]  Julian Togelius,et al.  Towards a Video Game Description Language , 2013, Artificial and Computational Intelligence in Games.

[24]  Tom Schaul,et al.  A video game description language for model-based or interactive learning , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[25]  K. Kamibayashi.  Motor Control and Learning , 2014 .