Towards Agent-Based Testing of 3D Games using Reinforcement Learning

Computer game is a billion-dollar industry and is booming. Testing games has been recognized as a difficult task, which mainly relies on manual playing and scripting based testing. With the advances in technologies, computer games have become increasingly more interactive and complex, thus play-testing using human participants alone has become unfeasible. In recent days, play-testing of games via autonomous agents has shown great promise by accelerating and simplifying this process. Reinforcement Learning solutions have the potential of complementing current scripted and automated solutions by learning directly from playing the game without the need of human intervention. This paper presented an approach based on reinforcement learning for automated testing of 3D games. We make use of the notion of curiosity as a motivating factor to encourage an RL agent to explore its environment. The results from our exploratory study are promising and we have preliminary evidence that reinforcement learning can be adopted for automated testing of 3D games.

[1]  Gabriele Bavota,et al.  Using Reinforcement Learning for Load Testing of Video Games , 2022, 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE).

[2]  Martin A. Riedmiller,et al.  Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration , 2021, ArXiv.

[3]  Konrad Tollmar,et al.  Improving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agents , 2021, 2021 IEEE Conference on Games (CoG).

[4]  Fabio Petrillo,et al.  A Survey of Video Game Testing , 2021, 2021 IEEE/ACM International Conference on Automation of Software Test (AST).

[5]  Tanja E. J. Vos,et al.  Agent-based Testing of Extended Reality Systems , 2020, 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST).

[6]  Joakim Bergdahl,et al.  Augmenting Automated Game Testing with Deep Reinforcement Learning , 2020, 2020 IEEE Conference on Games (CoG).

[7]  Xuandong Li,et al.  Reinforcement learning based curiosity-driven testing of Android applications , 2020, ISSTA.

[8]  Jakub W. Pachocki,et al.  Dota 2 with Large Scale Deep Reinforcement Learning , 2019, ArXiv.

[9]  Lei Ma,et al.  Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning , 2019, 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[10]  Salima Hassas,et al.  A survey on intrinsic motivation in reinforcement learning , 2019, ArXiv.

[11]  Aysu Betin Can,et al.  Automated Video Game Testing Using Synthetic and Humanlike Agents , 2019, IEEE Transactions on Games.

[12]  Renée C. Bryce,et al.  Reinforcement learning for Android GUI testing , 2018, A-TEST@ESEC/SIGSOFT FSE.

[13]  Marc Pollefeys,et al.  Episodic Curiosity through Reachability , 2018, ICLR.

[14]  Alexei A. Efros,et al.  Large-Scale Study of Curiosity-Driven Learning , 2018, ICLR.

[15]  Pierre-Yves Oudeyer,et al.  Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration , 2018, ICLR.

[16]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[17]  Tom Schaul,et al.  StarCraft II: A New Challenge for Reinforcement Learning , 2017, ArXiv.

[18]  Alexei A. Efros,et al.  Curiosity-Driven Exploration by Self-Supervised Prediction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Marc G. Bellemare,et al.  Count-Based Exploration with Neural Density Models , 2017, ICML.

[20]  Filip De Turck,et al.  #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning , 2016, NIPS.

[21]  Guillaume Lample,et al.  Playing FPS Games with Deep Reinforcement Learning , 2016, AAAI.

[22]  Filip De Turck,et al.  VIME: Variational Information Maximizing Exploration , 2016, NIPS.

[23]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[24]  Tom Apperley.  Genre and game studies: Toward a critical approach to video game genres , 2006 .

[25]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[26]  P. Jaccard.  THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1 , 1912 .

[27]  Fitsum Meshesha Kifetew,et al.  Search-Based Automated Play Testing of Computer Games: A Model-Based Approach , 2021, SSBSE.

[28]  Tanja E. J. Vos,et al.  Aplib: Tactical Agents for Testing Computer Games , 2020, EMAS@AAMAS.

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  A. Dixit.  Optimization in Economic Theory , 1976 .