PCGRL: Procedural Content Generation via Reinforcement Learning

We investigate how reinforcement learning can be used to train level-designing agents. This represents a new approach to procedural content generation in games, where level design is framed as a game, and the content generator itself is learned. By seeing the design problem as a sequential task, we can use reinforcement learning to learn how to take the next action so that the expected final level quality is maximized. This approach can be used when few or no examples exist to train from, and the trained generator is very fast. We investigate three different ways of transforming two-dimensional level design problems into Markov decision processes and apply these to three game environments.

[1]  Julian Togelius,et al.  Search-Based Procedural Content Generation: A Taxonomy and Survey , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[2]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[3]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[4]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[5]  Julian Togelius,et al.  Linear levels through n-grams , 2014, MindTrek.

[6]  Zhengxing Chen,et al.  Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[7]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[8]  E. F. Codd,et al.  Cellular automata , 1968 .

[9]  Michael Mateas,et al.  Super Mario as a String: Platformer Level Generation Via LSTMs , 2016, DiGRA/FDG.

[10]  Julian Togelius,et al.  Procedural Content Generation via Machine Learning (PCGML) , 2017, IEEE Transactions on Games.

[11]  Julian Togelius,et al.  General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms , 2018, IEEE Transactions on Games.

[12]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[13]  Sam Earle.  Using Fractal Neural Networks to Play SimCity 1 and Conway's Game of Life at Variable Scales , 2020, ArXiv.

[14]  Simon M. Lucas,et al.  Evolving mario levels in the latent space of a deep convolutional generative adversarial network , 2018, GECCO.

[15]  Julian Togelius,et al.  Deep Learning for Video Game Playing , 2017, IEEE Transactions on Games.

[16]  Julian Togelius,et al.  Procedural Content Generation in Games , 2016, Computational Synthesis and Creative Systems.

[17]  Quoc V. Le,et al.  Neural Architecture Search with Reinforcement Learning , 2016, ICLR.

[18]  Matthew Guzdial,et al.  Co-Creative Level Design via Machine Learning , 2018, AIIDE Workshops.

[19]  Julian Togelius,et al.  Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation , 2018, 1806.10729.

[20]  Antonios Liapis,et al.  Mixed-initiative co-creativity , 2014, FDG.

[21]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[22]  Edmund K. Burke,et al.  The Genetic and Evolutionary Computation Conference , 2011 .

[23]  Santiago Ontañón,et al.  Experiments in map generation using Markov chains , 2014, FDG.

[24]  Julian Togelius,et al.  A procedural procedural level generator generator , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[25]  Julian Togelius,et al.  Tree Search vs Optimization Approaches for Map Generation , 2019, ArXiv.