Illuminating Diverse Neural Cellular Automata for Level Generation

We present a method of generating a collection of neural cellular automata (NCA) to design video game levels. While NCAs have so far only been trained via supervised learning, we present a quality diversity (QD) approach to generating a collection of NCA level generators. By framing the problem as a QD problem, our approach can train diverse level generators, whose output levels vary based on aesthetic or functional criteria. To efficiently generate NCAs, we train generators via Covariance Matrix Adaptation MAP-Elites (CMA-ME), a quality diversity algorithm which specializes in continuous search spaces. We apply our new method to generate level generators for several 2D tile-based games: a maze game, Sokoban, and Zelda. Our results show that CMA-ME can generate small NCAs that are diverse yet capable, often satisfying complex solvability criteria for deterministic agents. We compare against a Compositional Pattern-Producing Network (CPPN) baseline trained to produce diverse collections of generators and show that the NCA representation yields a better exploration of level-space.

[1]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[2]  Josef Sivic,et al.  Evolving Structures in Complex Systems , 2019, 2019 IEEE Symposium Series on Computational Intelligence (SSCI).

[3]  Julian Togelius,et al.  Procedural Content Generation via Machine Learning (PCGML) , 2017, IEEE Transactions on Games.

[4]  Kenneth O. Stanley,et al.  Confronting the Challenge of Quality Diversity , 2015, GECCO.

[5]  Jimmy Secretan,et al.  Picbreeder: evolving pictures collaboratively online , 2008, CHI.

[6]  Kenneth O. Stanley,et al.  Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks , 2016, GECCO.

[7]  Julian Togelius,et al.  Video Game Level Repair via Mixed Integer Linear Programming , 2020, AIIDE.

[8]  Sam Earle.  Using Fractal Neural Networks to Play SimCity 1 and Conway's Game of Life at Variable Scales , 2020, ArXiv.

[9]  Julian Togelius,et al.  Learning Controllable Content Generators , 2021, 2021 IEEE Conference on Games (CoG).

[10]  Antoine Cully,et al.  Quality-Diversity Optimization: a novel branch of stochastic optimization , 2020, Black Box Optimization, Machine Learning, and No-Free Lunch Theorems.

[11]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[12]  Jean-Baptiste Mouret,et al.  Illuminating search spaces by mapping elites , 2015, ArXiv.

[13]  Juergen Branke,et al.  BOP-Elites, a Bayesian Optimisation algorithm for Quality-Diversity search , 2020, ArXiv.

[14]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[15]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[16]  Janet Wiles,et al.  Combining Influence Maps and Cellular Automata for Reactive Game Agents , 2005, IDEAL.

[17]  Julian Togelius,et al.  Covariance matrix adaptation for the rapid illumination of behavior space , 2020, GECCO.

[18]  Julian Togelius,et al.  PCGRL: Procedural Content Generation via Reinforcement Learning , 2020, AAAI 2020.

[19]  Kenneth O. Stanley,et al.  Quality Diversity: A New Frontier for Evolutionary Computation , 2016, Front. Robot. AI.

[20]  Julian Togelius,et al.  Mixed-Initiative Level Design with RL Brush , 2020, EvoMUSART.

[21]  Ettore Randazzo,et al.  Self-Organising Textures , 2021 .

[22]  Sebastian Risi,et al.  Growing 3D Artefacts and Functional Machines with Neural Cellular Automata , 2021, The 2021 Conference on Artificial Life.

[23]  Michael Carbin,et al.  The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks , 2018, ICLR.

[24]  Max Jaderberg,et al.  Open-Ended Learning Leads to Generally Capable Agents , 2021, ArXiv.

[25]  Kenneth O. Stanley,et al.  Evolving a diversity of virtual creatures through novelty search and local competition , 2011, GECCO '11.

[26]  Garrett T. Kenyon,et al.  It's Hard for Neural Networks to Learn the Game of Life , 2020, 2021 International Joint Conference on Neural Networks (IJCNN).

[27]  Gregory Shakhnarovich,et al.  FractalNet: Ultra-Deep Neural Networks without Residuals , 2016, ICLR.

[28]  J. Clune,et al.  Scaling MAP-Elites to deep neuroevolution , 2020, GECCO.

[29]  David Ha,et al.  Generating Abstract Patterns with TensorFlow , 2016 .

[30]  Stephen Wolfram,et al.  Cellular automata as models of complexity , 1984, Nature.

[31]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[32]  Kenneth O. Stanley,et al.  Compositional Pattern Producing Networks : A Novel Abstraction of Development , 2007 .

[33]  Sushil J. Louis,et al.  Procedural maze level generation with evolutionary cellular automata , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[34]  Kenneth O. Stanley,et al.  Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents , 2017, NeurIPS.

[35]  Maxwell Nye,et al.  Are Efficient Deep Representations Learnable? , 2018, ICLR.

[36]  Julian Togelius,et al.  Cellular automata for real-time generation of infinite cave levels , 2010, PCGames@FDG.

[37]  Pierre-Yves Oudeyer,et al.  Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems , 2020, ICLR.

[38]  Michael Levin,et al.  Growing Neural Cellular Automata , 2020, Distill.