Neurosymbolic Map Generation with VQ-VAE and WFC

We introduce a hybrid neural + symbolic approach to map generation that combines neural discrete representation learning with symbolic constraint solving methods. In application to WarCraft II and Super Metroid map designs, we show how a vocabulary of directly manipulable latent tiles can be inferred from the raw pixels of design training data. Despite working with a very small tile vocabulary, our method is able to express a very large effective set of unique tiles at the level of pixel appearances. This work shows new ways of combining generative methods, resulting in directly controllable generators for domains that are primarily specified only by visual design examples.

[1]  Michael Mateas,et al.  Tanagra: a mixed-initiative level design tool , 2010, FDG.

[2]  Oriol Vinyals,et al.  Neural Discrete Representation Learning , 2017, NIPS.

[3]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[4]  I. Horswill.  Imaginarium: A Tool for Casual Constraint-Based PCG , 2019 .

[5]  Luis C. Lamb,et al.  Neurosymbolic AI: the 3rd wave , 2020, Artificial Intelligence Review.

[6]  Michael Mateas,et al.  Answer Set Programming for Procedural Content Generation: A Design Space Approach , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[7]  Adam M. Smith,et al.  Addressing the fundamental tension of PCGML with discriminative learning , 2018, FDG.

[8]  Jaakko Lehtinen,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zoran Popovic,et al.  Quantifying over play: Constraining undesirable solutions in puzzle design , 2013, FDG.

[10]  Adam M. Smith,et al.  WaveFunctionCollapse is constraint solving in the wild , 2017, FDG.

[11]  Santiago Ontañón,et al.  A Hierarchical MdMC Approach to 2D Video Game Map Generation , 2015, AIIDE.

[12]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[13]  Ilya Sutskever,et al.  Zero-Shot Text-to-Image Generation , 2021, ICML.

[14]  Julian Togelius,et al.  Deep learning for procedural content generation , 2020, Neural Computing and Applications.

[15]  Michael Mateas,et al.  Super Mario as a String: Platformer Level Generation Via LSTMs , 2016, DiGRA/FDG.

[16]  Patrick Esser,et al.  Taming Transformers for High-Resolution Image Synthesis , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ian Horswill,et al.  Rolling Your Own Finite-Domain Constraint Solver , 2015 .

[18]  Simon M. Lucas,et al.  Evolving mario levels in the latent space of a deep convolutional generative adversarial network , 2018, GECCO.