Game Engine Learning from Video

Intelligent agents need to be able to make predictions about their environment. In this work we present a novel approach to learn a forward simulation model via simple search over pixel input. We make use of a video game, Super Mario Bros., as an initial test of our approach as it represents a physics system that is significantly less complex than reality. We demonstrate the significant improvement of our approach in predicting future states compared with a baseline CNN and apply the learned model to train a game playing agent. Thus we evaluate the algorithm in terms of the accuracy and value of its output model.

[1]  J. Meigs,et al.  WHO Technical Report , 1954, The Yale Journal of Biology and Medicine.

[2]  C. Gettys,et al.  Hypothesis plausibility and hypothesis generation , 1979 .

[3]  ScienceDirect.  Organizational behavior and human performance , 1984 .

[4]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[5]  Rodney A. Brooks,et al.  Cambrian Intelligence: The Early History of the New AI , 1999 .

[6]  Erik T. Mueller,et al.  Open Mind Common Sense: Knowledge Acquisition from the General Public , 2002, OTM.

[7]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[8]  Julian Togelius,et al.  Mario AI competition , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[9]  Julian Togelius,et al.  The 2009 Mario AI Competition , 2010, IEEE Congress on Evolutionary Computation.

[10]  Nicholas J. Butko,et al.  Active perception , 2010 .

[11]  Julian Togelius,et al.  A Procedural Critique of Deontological Reasoning , 2011, DiGRA Conference.

[12]  Kari Pulli,et al.  Realtime Computer Vision with OpenCV , 2012, ACM Queue.

[13]  Kari Pulli,et al.  Real-time computer vision with OpenCV , 2012, Commun. ACM.

[14]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[15]  Simon Colton,et al.  Mechanic Miner: Reflection-Driven Game Mechanic Discovery and Level Design , 2013, EvoApplications.

[16]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[17]  Marc'Aurelio Ranzato,et al.  Video (language) modeling: a baseline for generative models of natural videos , 2014, ArXiv.

[18]  Mark O. Riedl,et al.  Automatic Game Design via Mechanic Generation , 2014, AAAI.

[19]  Joseph Corneli,et al.  Towards Generating Novel Games Using Conceptual Blending , 2021, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[20]  Alessandro Canossa,et al.  Towards a Procedural Evaluation Technique: Metrics for Level Design , 2015, FDG.

[21]  Santiago Ontañón,et al.  Automatic Learning of Combat Models for RTS Games , 2015, AIIDE.

[22]  Mark O. Riedl,et al.  Monte-Carlo Tree Search for Simulation-based Play Strategy Analysis , 2015, FDG.

[23]  Sanem Sariel,et al.  Learning Behaviors of and Interactions Among Objects Through Spatio–Temporal Reasoning , 2015, IEEE Transactions on Computational Intelligence and AI in Games.

[24]  Nathan R Sturtevant,et al.  Deep Static and Dynamic Level Analysis: A Study on Infinite Mario , 2016 .

[25]  Honglak Lee,et al.  Control of Memory, Active Perception, and Action in Minecraft , 2016, ICML.

[26]  M. Mateas,et al.  Proceduralist Readings, Procedurally , 2016 .

[27]  Danijar Hafner,et al.  Deep Reinforcement Learning From Raw Pixels in Doom , 2016, ArXiv.

[28]  Julian Togelius,et al.  Rules and Mechanics , 2016 .

[29]  Risto Miikkulainen,et al.  Object-Model Transfer in the General Video Game Domain , 2016, AIIDE.

[30]  Morteza Behrooz,et al.  What Does That ?-Block Do? Learning Latent Causal Affordances From Mario Play Traces , 2017, AAAI Workshops.

[31]  Guillaume Gravier,et al.  One-Step Time-Dependent Future Video Frame Prediction with a Convolutional Encoder-Decoder Neural Network , 2016, ICIAP.

[32]  Julian Togelius,et al.  Procedural Content Generation via Machine Learning (PCGML) , 2017, IEEE Transactions on Games.

[33]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).