Enhancing the Monte Carlo Tree Search Algorithm for Video Game Testing

In this paper, we study the effects of several Monte Carlo Tree Search (MCTS) modifications for video game testing. Although MCTS modifications are highly studied in game playing, their impacts on finding bugs are blank. We focused on bug finding in our previous study where we introduced synthetic and human-like test goals and we used these test goals in Sarsa and MCTS agents to find bugs. In this study, we extend the MCTS agent with several modifications for game testing purposes. Furthermore, we present a novel tree reuse strategy. We experiment with these modifications by testing them on three testbed games, four levels each, that contain 45 bugs in total. We use the General Video Game Artificial Intelligence (GVG-AI) framework to create the testbed games and collect 427 human tester trajectories using the GVG-AI framework. We analyze the proposed modifications in three parts: we evaluate their effects on bug finding performances of agents, we measure their success under two different computational budgets, and we assess their effects on human-likeness of the human-like agent. Our results show that MCTS modifications improve the bug finding performance of the agents.

[1]  Mark H. M. Winands,et al.  Time Management for Monte Carlo Tree Search , 2016, IEEE Transactions on Computational Intelligence and AI in Games.

[2]  Mark H. M. Winands,et al.  Real-Time Monte Carlo Tree Search in Ms Pac-Man , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[3]  Simon M. Lucas,et al.  Knowledge-based fast evolutionary MCTS for general video game playing , 2014, 2014 IEEE Conference on Computational Intelligence and Games.

[4]  Julian Togelius,et al.  Ieee Transactions on Computational Intelligence and Ai in Games the 2014 General Video Game Playing Competition , 2022 .

[5]  Levente Kocsis,et al.  Transpositions and move groups in Monte Carlo tree search , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[6]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[7]  Julian Togelius,et al.  AI-Assisted Game Debugging with Cicero , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[8]  Peter I. Cowling,et al.  Information capture and reuse strategies in Monte Carlo Tree Search, with applications to games of hidden information , 2014, Artif. Intell..

[9]  Francisco S. Melo,et al.  Monte Carlo tree search experiments in hearthstone , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[10]  Ji-Hoon Kang,et al.  Scenario-Based Approach for Blackbox Load Testing of Online Game Servers , 2010, 2010 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery.

[11]  Julian Togelius,et al.  Modifying MCTS for Human-Like General Video Game Playing , 2016, IJCAI.

[12]  Mark H. M. Winands,et al.  N-Grams and the Last-Good-Reply Policy Applied in General Game Playing , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[13]  Aysu Betin Can,et al.  Automated Video Game Testing Using Synthetic and Humanlike Agents , 2019, IEEE Transactions on Games.

[14]  A. Sima Etaner-Uyar,et al.  Monte Carlo tree search with temporal-difference learning for general video game playing , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[15]  Jong-Kook Kim,et al.  Enhancing Monte Carlo Tree Search for Playing Hearthstone , 2019, 2019 IEEE Conference on Games (CoG).

[16]  George Konidaris,et al.  An Analysis of Monte Carlo Tree Search , 2017, AAAI.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Muhammad Zohaib Z. Iqbal,et al.  An automated model based testing approach for platform games , 2015, 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS).

[19]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[20]  Julian Togelius,et al.  Monte Mario: platforming with MCTS , 2014, GECCO.

[21]  Gerald Tesauro,et al.  Monte-Carlo simulation balancing , 2009, ICML '09.

[22]  H. Jaap van den Herik,et al.  Single-Player Monte-Carlo Tree Search , 2008, Computers and Games.

[23]  Peter I. Cowling,et al.  Memory Bounded Monte Carlo Tree Search , 2017, AIIDE.

[24]  Rainer Malaka,et al.  Automated Game Testing with ICARUS: Intelligent Completion of Adventure Riddles via Unsupervised Solving , 2017, CHI PLAY.

[25]  Mark J. Nelson,et al.  Investigating vanilla MCTS scaling on the GVG-AI game corpus , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[26]  Michail Ostrowski,et al.  Automated Regression Testing within Video Game Development , 2013 .

[27]  Simon M. Lucas,et al.  Multiobjective Monte Carlo Tree Search for Real-Time Games , 2015, IEEE Transactions on Computational Intelligence and AI in Games.

[28]  Julian Togelius,et al.  Investigating MCTS modifications in general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[29]  Tom Schaul,et al.  An Extensible Description Language for Video Games , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[30]  Pedro Pablo Gómez-Martín,et al.  An approach to automated videogame beta testing , 2024, Entertain. Comput..

[31]  Yew-Soon Ong,et al.  Specialization of a UCT-Based General Game Playing Program to Single-Player Games , 2016, IEEE Transactions on Computational Intelligence and AI in Games.

[32]  Dennis J. N. J. Soemers,et al.  Enhancements for real-time Monte-Carlo Tree Search in General Video Game Playing , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[33]  Luiz Fernando Capretz,et al.  Computer games are serious business and so is their quality: particularities of software testing in game development from the perspective of practitioners , 2018, ESEM.

[34]  Martin Müller,et al.  Memory-Augmented Monte Carlo Tree Search , 2018, AAAI.