Augmenting Automated Game Testing with Deep Reinforcement Learning

General game testing relies on the use of human play testers, play test scripting, and prior knowledge of areas of interest to produce relevant test data. Using deep reinforcement learning (DRL), we introduce a self-learning mechanism to the game testing framework. With DRL, the framework is capable of exploring and/or exploiting the game mechanics based on a user-defined, reinforcing reward signal. As a result, test coverage is increased and unintended game play mechanics, exploits and bugs are discovered in a multitude of game types. In this paper, we show that DRL can be used to increase test coverage, find exploits, test map difficulty, and to detect common problems that arise in the testing of first-person shooter (FPS) games.

[1]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[2]  Razvan Pascanu,et al.  Learning to Navigate in Complex Environments , 2016, ICLR.

[3]  Igor Mordatch,et al.  Emergent Tool Use From Multi-Agent Autocurricula , 2019, ICLR.

[4]  Alexei A. Efros,et al.  Curiosity-Driven Exploration by Self-Supervised Prediction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Alper Sen,et al.  QBE: QLearning-Based Exploration of Android Applications , 2018, 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST).

[6]  Mark O. Riedl,et al.  Automatic playtesting for game parameter tuning via active learning , 2019, FDG.

[7]  Kazi A. Zaman,et al.  Winning Isn ’ t Everything : Training Agents to Playtest Modern Games , 2019 .

[8]  Andrew Nealen,et al.  Exploring Game Space Using Survival Analysis , 2015, FDG.

[9]  Kristoffer Sjöö,et al.  Imitation Learning with Concurrent Actions in 3D Games , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[10]  Stefan Freyr Gudmundsson,et al.  Human-Like Playtesting with Deep Learning , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[11]  Guillaume Lample,et al.  Playing FPS Games with Deep Reinforcement Learning , 2016, AAAI.

[12]  Lei Ma,et al.  Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning , 2019, 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).