A Dynamic Balanced Level Generator for Video Games based on Deep Convolutional Generative Adversarial Networks

In the gaming industry, creating well-balanced games is one of the major challenges developers are currently facing. Balance in games has different meanings depending on the game type. But, most of the existing definitions are esteemed from the flow theory. Flow theory in video games is stating that the level of challenge existing in the game must be neither too easy nor too difficult for the player. Games that are not balanced will have a high churn rate and will suffer in terms of monetization. Hence, nowadays a trending research area is focused on establishing mechanisms to create automatic balance in an algorithmic way. In this research, we have used generative adversarial networks (GANs) to automatically create balanced levels. In the proposed work, a level of a 2D platformer game is fed to the network. Finally, the network automatically generates new balanced levels and the levels are checked to see if they have the game’s minimum necessary requirements and also to check if they can be solved by the reinforcement learning agent. In the series of performed evaluations, it is shown that after the training process, the proposed approach is capable of generating levels that are well-balanced with considerable accuracy.

[1]  Jon McCormack,et al.  Interactive evolution of L-System grammars for computer graphics modelling , 1993 .

[2]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[3]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Ernest Adams,et al.  Fundamentals of Game Design , 2006 .

[6]  Farès Belhadj,et al.  Terrain modeling: a constrained fractal model , 2007, AFRIGRAPH '07.

[7]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[8]  Kazunori Mizuno,et al.  Constructive generation of very hard 3-colorability instances , 2008, Discret. Appl. Math..

[9]  Georgios N. Yannakakis,et al.  Real-time challenge balance in an RTS game using rtNEAT , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[10]  Jesse Schell,et al.  The Art of Game Design: A book of lenses , 2019 .

[11]  C. R. Snyder,et al.  Oxford Handbook of Positive Psychology , 2009 .

[12]  Francisco Herrera,et al.  A Survey on the Application of Genetic Programming to Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Yuan Gao,et al.  Dynamic difficulty adjustment of game AI for video game Dead-End , 2010, The 3rd International Conference on Information Sciences and Interaction Sciences.

[14]  Julian Togelius,et al.  Search-Based Procedural Content Generation: A Taxonomy and Survey , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[15]  Jim Whitehead,et al.  Weapon design patterns in shooter games , 2012, DPG '12.

[16]  Zoran Popovic,et al.  Evaluating Competitive Game Balance with Restricted Play , 2012, AIIDE.

[17]  Alexandru Iosup,et al.  Procedural content generation for games: A survey , 2013, TOMCCAP.

[18]  Jose L. Salmeron,et al.  A Review of Fuzzy Cognitive Maps Research During the Last Decade , 2013, IEEE Transactions on Fuzzy Systems.

[19]  Marjan Mernik,et al.  Exploration and exploitation in evolutionary algorithms: A survey , 2013, CSUR.

[20]  Santiago Ontañón,et al.  PSMAGE: Balanced map generation for StarCraft , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[21]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[22]  Shaun Bangay,et al.  Generating an attribute space for analyzing balance in single unit RTS game combat , 2014, 2014 IEEE Conference on Computational Intelligence and Games.

[23]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[24]  Luiz Chaimowicz,et al.  Dynamic Difficulty Adjustment through an Adaptive AI , 2015, 2015 14th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames).

[25]  Esteban Walter Gonzalez Clua,et al.  Dynamic Game Difficulty Balancing in Real Time Using Evolutionary Fuzzy Cognitive Maps , 2015, 2015 14th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames).

[26]  Antonios Liapis,et al.  Multi-Level Evolution of Shooter Levels , 2021, AIIDE.

[27]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[28]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[29]  Santiago Ontañón,et al.  The VGLC: The Video Game Level Corpus , 2016, ArXiv.

[30]  Bhojan Anand,et al.  Game balancing with ecosystem mechanism , 2016, 2016 International Conference on Data Mining and Advanced Computing (SAPIENCE).

[31]  Joshua B. Tenenbaum,et al.  Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation , 2016, NIPS.

[32]  Julian Togelius,et al.  Procedural Content Generation in Games , 2016, Computational Synthesis and Creative Systems.

[33]  Jin Liu,et al.  Multi-scale multi-class conditional generative adversarial network for handwritten character generation , 2017, The Journal of Supercomputing.

[34]  Guillaume Bosc,et al.  A Pattern Mining Approach to Study Strategy Balance in RTS Games , 2017, IEEE Transactions on Computational Intelligence and AI in Games.

[35]  Antonios Liapis,et al.  Learning the patterns of balance in a multi-player shooter game , 2017, FDG.

[36]  Yi-Hsuan Yang,et al.  MidiNet: A Convolutional Generative Adversarial Network for Symbolic-Domain Music Generation , 2017, ISMIR.

[37]  Riccardo Poli,et al.  Automated Game Balancing in Ms PacMan and StarCraft Using Evolutionary Algorithms , 2017, EvoApplications.

[38]  Shaun Bangay,et al.  Orthogonal analysis of StarCraft II for game balance , 2017, ACSW.

[39]  Julian Togelius,et al.  DeepMasterPrint: Fingerprint Spoofing via Latent Variable Evolution , 2017 .

[40]  Anastasios I. Dounis,et al.  Design of a Fuzzy Cognitive Maps variable-load energy management system for autonomous PV-reverse osmosis desalination systems: A simulation survey , 2017 .

[41]  Riccardo Poli,et al.  Evolving a designer-balanced neural network for Ms PacMan , 2017, 2017 9th Computer Science and Electronic Engineering (CEEC).

[42]  Yang Liu,et al.  Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening , 2016, ICLR.

[43]  S. Shankar Sastry,et al.  Markov Decision Process Routing Games , 2017, 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS).

[44]  Sanjay Singh,et al.  Fitness function to find game equilibria using genetic algorithms , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[45]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Yong Man Ro,et al.  Dynamics Transfer GAN: Generating Video by Transferring Arbitrary Temporal Dynamics from a Source Video to a Single Target Image , 2017, ArXiv.

[47]  Kirill Krinkin,et al.  2D SLAM quality evaluation methods , 2017, 2017 21st Conference of Open Innovations Association (FRUCT).

[48]  Simon M. Lucas,et al.  Evolving mario levels in the latent space of a deep convolutional generative adversarial network , 2018, GECCO.

[49]  Samy Bengio,et al.  A Study on Overfitting in Deep Reinforcement Learning , 2018, ArXiv.

[50]  Edoardo Giacomello,et al.  DOOM Level Generation Using Generative Adversarial Networks , 2018, 2018 IEEE Games, Entertainment, Media Conference (GEM).

[51]  Jakub W. Pachocki,et al.  Dota 2 with Large Scale Deep Reinforcement Learning , 2019, ArXiv.

[52]  Gethin Norman,et al.  Balancing Turn-Based Games With Chained Strategy Generation , 2019, IEEE Transactions on Games.