Co-creating Platformer Levels with Constrained Adversarial Networks

Given the success of deep generative models in many creative tasks, it is natural to ask how to best leverage them to support human designers. We study this problem in the context of mixed-initiative design of platformer levels, a paradigmatic co-creative task. This setting is especially challenging, because – like all functional content – platformer levels must satisfy complex validity constraints, like coherency and playability. We explore mixed-initiative interaction with constrained adversarial networks (CANs), a class of deep generative models that synthesize structures satisfying one or more validity constraints. As such, CANs can be used to complete user-supplied partial levels while retaining full control of the constraints to be applied. We go one step beyond, and consider the issue of customizing a pre-trained CAN to some target design task at hand and to the designer’s preferences. We discuss how to achieve this by combining CANs with coactive learning, a very natural mixed-initiate interaction protocol that acquires the necessary supervision from the designer in a transparent manner. Finally, we illustrate how to extend coactive learning to acquire informative supervision in the form of interpretable constraints.

[1]  K. Kersting,et al.  Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Julian Togelius,et al.  Deep learning for procedural content generation , 2020, Neural Computing and Applications.

[3]  Paolo Burelli,et al.  Procedural Content Generation of Puzzle Games using Conditional Generative Adversarial Networks , 2020, FDG.

[4]  Nicu Sebe,et al.  Describe What to Change: A Text-guided Unsupervised Image-to-image Translation Approach , 2020, ACM Multimedia.

[5]  Paolo Morettin,et al.  Efficient Generation of Structured Objects with Constrained Adversarial Networks , 2020, NeurIPS.

[6]  Jacob Schrum,et al.  CPPN2GAN: combining compositional pattern producing networks and GANs for large-scale pattern generation , 2020, GECCO.

[7]  Pegah Karimi,et al.  Creative sketching partner: an analysis of human-AI co-creativity , 2020, IUI.

[8]  Kristian Kersting,et al.  Making deep neural networks right for the right scientific reasons by interacting with their explanations , 2020, Nat. Mach. Intell..

[9]  Julian Togelius,et al.  Bootstrapping Conditional GANs for Video Game Level Generation , 2019, 2020 IEEE Conference on Games (CoG).

[10]  Jianfeng Gao,et al.  Sequential Attention GAN for Interactive Image Editing , 2018, ACM Multimedia.

[11]  Yoshua Bengio,et al.  MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis , 2019, NeurIPS.

[12]  Angelo Montanari,et al.  Synthesis of LTL Formulas from Natural Language Texts: State of the Art and Research Directions , 2019, TIME.

[13]  Winston K. G. Seah,et al.  A Hybrid Online Offline System for Network Anomaly Detection , 2019, 2019 28th International Conference on Computer Communication and Networks (ICCCN).

[14]  Eric Heim,et al.  Constrained Generative Adversarial Networks for Interactive Image Generation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Adam M. Smith,et al.  Addressing the fundamental tension of PCGML with discriminative learning , 2018, FDG.

[16]  D. Boller Tell , 2018, Reader Superhelden.

[17]  Yoshua Bengio,et al.  Keep Drawing It: Iterative language-based image generation and editing , 2018, ArXiv.

[18]  Luc De Raedt,et al.  Learning Constraints From Examples , 2018, AAAI.

[19]  Noam Brown,et al.  Superhuman AI for heads-up no-limit poker: Libratus beats top professionals , 2018, Science.

[20]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Andrea Passerini,et al.  Constructive Preference Elicitation , 2018, Front. Robot. AI.

[22]  Guy Van den Broeck,et al.  A Semantic Loss Function for Deep Learning with Symbolic Knowledge , 2017, ICML.

[23]  Julian Togelius,et al.  Procedural Content Generation via Machine Learning (PCGML) , 2017, IEEE Transactions on Games.

[24]  Kevin Waugh,et al.  DeepStack: Expert-level artificial intelligence in heads-up no-limit poker , 2017, Science.

[25]  Andrea Passerini,et al.  Coactive Critiquing: Elicitation of Preferences and Features , 2016, AAAI.

[26]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[27]  Santiago Ontañón,et al.  An Approach to Domain Transfer in Procedural Content Generation of Two-Dimensional Videogame Levels , 2021, AIIDE.

[28]  Matthew Guzdial,et al.  Game Level Generation from Gameplay Videos , 2021, AIIDE.

[29]  Gabriella Pigozzi,et al.  Preferences in artificial intelligence , 2016, Annals of Mathematics and Artificial Intelligence.

[30]  Thorsten Joachims,et al.  Coactive Learning , 2015, J. Artif. Intell. Res..

[31]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[32]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[33]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[34]  Michael Mateas,et al.  Tanagra: a mixed-initiative level design tool , 2010, FDG.

[35]  John Langford,et al.  Slow Learners are Fast , 2009, NIPS.

[36]  Matthias Scheutz,et al.  What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution , 2009, 2009 IEEE International Conference on Robotics and Automation.

[37]  Krzysztof Z. Gajos,et al.  Preference elicitation for interface optimization , 2005, UIST.

[38]  Pierre Marquis,et al.  A Knowledge Compilation Map , 2002, J. Artif. Intell. Res..