Dungeons & Replicants: Automated Game Balancing via Deep Player Behavior Modeling

Balancing the options available to players in a way that ensures rich variety and viability is a vital factor for the success of any video game, and particularly competitive multiplayer games. Traditionally, this balancing act requires extensive periods of expert analysis, play testing and debates. While automated gameplay is able to predict outcomes of parameter changes, current approaches mainly rely on heuristic or optimal strategies to generate agent behavior. In this paper, we demonstrate the use of deep player behavior models to represent a player population (n = 213) of the massively multiplayer online role-playing game Aion, which are used, in turn, to generate individual agent behaviors. Results demonstrate significant balance differences in opposing enemy encounters and show how these can be regulated. Moreover, the analytic methods proposed are applied to identify the balance relationships between classes when fighting against each other, reflecting the original developers’ design.

[1]  Markus Schatten,et al.  Automated MMORPG Testing - An Agent-Based Approach , 2017, PAAMS.

[2]  Zoran Popovic,et al.  Evaluating Competitive Game Balance with Restricted Play , 2012, AIIDE.

[3]  Sylvain Hallé,et al.  Automated Bug Finding in Video Games: A Case Study for Runtime Monitoring , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[4]  Sander Bakkes,et al.  Automated game balancing of asymmetric video games , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[5]  Riccardo Poli,et al.  Automated Game Balancing in Ms PacMan and StarCraft Using Evolutionary Algorithms , 2017, EvoApplications.

[6]  Stefan Freyr Gudmundsson,et al.  Human-Like Playtesting with Deep Learning , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[7]  Maarten Löffler,et al.  Automated puzzle difficulty estimation , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[8]  Pedro Pablo Gómez-Martín,et al.  An approach to automated videogame beta testing , 2024, Entertain. Comput..

[9]  Julian Togelius,et al.  Evolving card sets towards balancing dominion , 2012, 2012 IEEE Congress on Evolutionary Computation.

[10]  Rainer Malaka,et al.  Procedural Content Generation in Competitive Multiplayer Platform Games , 2019, ICEC-JCSG.

[11]  Simon Colton,et al.  Semi-automated level design via auto-playtesting for handheld casual game creation , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[12]  Julian Togelius,et al.  Automatic generation and analysis of physics-based puzzle games , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[13]  Brian M. Slator,et al.  Crushinator: A framework towards game-independent testing , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[14]  Muhammad Zohaib Z. Iqbal,et al.  An automated model based testing approach for platform games , 2015, 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS).

[15]  Jörg Denzinger,et al.  Evolutionary behavior testing of commercial computer games , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[16]  Rainer Malaka,et al.  Automated Game Testing with ICARUS: Intelligent Completion of Adventure Riddles via Unsupervised Solving , 2017, CHI PLAY.

[17]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[18]  Noah Wardrip-Fruin,et al.  Mining game statistics from web services: a World of Warcraft armory case study , 2010, FDG.

[19]  Rainer Malaka,et al.  Bot or not? User Perceptions of Player Substitution with Deep Player Behavior Models , 2020, CHI.

[20]  Juan Julián Merelo Guervós,et al.  Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone , 2018, Knowl. Based Syst..

[21]  Cor-Paul Bezemer,et al.  Studying the urgent updates of popular games on the Steam platform , 2017, Empirical Software Engineering.

[22]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[23]  Julian Togelius,et al.  AI as Evaluator: Search Driven Playtesting of Modern Board Games , 2017, AAAI Workshops.

[24]  Daniele Loiacono,et al.  Player Modeling , 2013, Artificial and Computational Intelligence in Games.

[25]  Rainer Malaka,et al.  Deep Player Behavior Models: Evaluating a Novel Take on Dynamic Difficulty Adjustment , 2019, CHI Extended Abstracts.

[26]  Rainer Malaka,et al.  Enemy Within: Long-term Motivation Effects of Deep Player Behavior Models for Dynamic Difficulty Adjustment , 2020, CHI.

[27]  Gang Xiao,et al.  Semi-Automated Gameplay Analysis by Machine Learning , 2005, AIIDE.

[28]  Christian Bird,et al.  "What Went Right and What Went Wrong": An Analysis of 155 Postmortems from Game Development , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[29]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[30]  Demis Hassabis,et al.  Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm , 2017, ArXiv.

[31]  Anne Sullivan,et al.  An inclusive view of player modeling , 2011, FDG.

[32]  Nachiappan Nagappan,et al.  Empirical analysis of user data in game software development , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[33]  Sushil J. Louis,et al.  Using coevolution to understand and validate game balance in continuous games , 2008, GECCO '08.

[34]  Lei Ma,et al.  Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning , 2019, 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[35]  Julian Togelius,et al.  Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics , 2018, IEEE Transactions on Games.

[36]  Julian Togelius,et al.  Artificial Intelligence and Games , 2018, Springer International Publishing.

[37]  Rainer Malaka,et al.  Towards Deep Player Behavior Models in MMORPGs , 2018, CHI PLAY.