General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms

General video game playing aims at designing an agent that is capable of playing multiple video games with no human intervention. In 2014, the General Video Game Artificial Intelligence (GVGAI) competition framework was created and released with the purpose of providing researchers a common open-source and easy-to-use platform for testing their artificial intelligence (AI) methods with potentially infinity of games created using the video game description language (VGDL). The framework has been expanded into several tracks during the last few years to meet the demands of different research directions. The agents are required either to play multiple unknown games with or without access to game simulations, or to design new game levels or rules. This survey paper presents the VGDL, the GVGAI framework, existing tracks, and reviews the wide use of GVGAI framework in research, education, and competitions five years after its birth. A future plan of framework improvements is also described.

[1]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[2]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Steven Orla Kimbrough,et al.  On a Feasible-Infeasible Two-Population (FI-2Pop) genetic algorithm for constrained optimization: Distance tracing and no free lunch , 2008, Eur. J. Oper. Res..

[5]  Philip Hingston.  The 2K BotPrize , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[6]  Julian Togelius,et al.  The 2009 Simulated Car Racing Championship , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[7]  Julian Togelius,et al.  The Mario AI Benchmark and Competitions , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[8]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[9]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[10]  Julian Togelius,et al.  The turing test track of the 2012 Mario AI Championship: Entries and evaluation , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[11]  Tom Schaul,et al.  A video game description language for model-based or interactive learning , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[12]  Julian Togelius,et al.  Towards a Video Game Description Language , 2013, Artificial and Computational Intelligence in Games.

[13]  Risto Miikkulainen,et al.  General Video Game Playing , 2013, Artificial and Computational Intelligence in Games.

[14]  Simon M. Lucas,et al.  Knowledge-based fast evolutionary MCTS for general video game playing , 2014, 2014 IEEE Conference on Computational Intelligence and Games.

[15]  Tom Schaul,et al.  An Extensible Description Language for Video Games , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[16]  Julian Togelius,et al.  Investigating MCTS modifications in general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[17]  T. Schuster.  MCTS BASED AGENT FOR GENERAL VIDEO GAMES , 2015 .

[18]  Marc Ebner,et al.  A strongly typed GP-based video game player , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[19]  J. G. van Eeden.  Analysing And Improving The Knowledge-based Fast Evolutionary MCTS Algorithm , 2015 .

[20]  Ruck Thawonmas,et al.  Combining pathfmding algorithm with Knowledge-based Monte-Carlo tree search in general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[21]  Julian Togelius,et al.  General Video Game Evaluation Using Relative Algorithm Performance Profiles , 2015, EvoApplications.

[22]  Marc Ebner,et al.  A GP-based Video Game Player , 2015, GECCO.

[23]  Kyung-Joong Kim,et al.  MCTS with influence map for general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[24]  Graham Kendall,et al.  EnHiC: An enforced hill climbing based system for general game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[25]  Hector Geffner,et al.  Width-Based Planning for General Video-Game Playing , 2015, AIIDE.

[26]  J. Togelius,et al.  Discovering Unique Game Variants , 2015 .

[27]  Ruck Thawonmas,et al.  Biasing Monte-Carlo Rollouts with Potential Field in General Video Game Playing , 2015 .

[28]  Sanaz Mostaghim,et al.  Procedural level generation with answer set programming for general Video Game playing , 2015, 2015 7th Computer Science and Electronic Engineering Conference (CEEC).

[29]  Simon M. Lucas,et al.  Open Loop Search for General Video Game Playing , 2015, GECCO.

[30]  Julian Togelius,et al.  Towards generating arcade game rules with VGDL , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[31]  Simon M. Lucas,et al.  Neuroevolution for General Video Game Playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[32]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[33]  Joel Lehman and Risto Miikkulainen.  General Video Game Playing as a Benchmark for Human-Competitive AI , 2015 .

[34]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[35]  Julian Togelius,et al.  Analyzing the robustness of general video game playing agents , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[36]  J. Togelius,et al.  Evolving UCT Alternatives for General Video Game Playing , 2016 .

[37]  Sander Bakkes,et al.  Monte Carlo Tree Search with options for general video game playing , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[38]  Julian Togelius,et al.  General Video Game Level Generation , 2016, GECCO.

[39]  Moshe Sipper,et al.  Evolving Artificial General Intelligence for Video Game Controllers , 2016, GPTP.

[40]  Mike Preuss,et al.  MCTS/EA hybrid GVGAI players and game difficulty estimation , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[41]  Ruck Thawonmas,et al.  Position-based reinforcement learning biased MCTS for General Video Game Playing , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[42]  Mark J. Nelson,et al.  Investigating vanilla MCTS scaling on the GVG-AI game corpus , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[43]  Julian Togelius,et al.  Matching Games and Algorithms for General Video Game Playing , 2021, AIIDE.

[44]  Julian Togelius,et al.  Modifying MCTS for Human-Like General Video Game Playing , 2016, IJCAI.

[45]  Julian Togelius,et al.  Hyper-heuristic general video game playing , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[46]  Simon M. Lucas,et al.  Ms. Pac-Man Versus Ghost Team CIG 2016 competition , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[47]  Simon M. Lucas,et al.  General Video Game for 2 players: Framework and competition , 2016, 2016 8th Computer Science and Electronic Engineering (CEEC).

[48]  Patrick M. Pilarski,et al.  True Online Temporal-Difference Learning , 2015, J. Mach. Learn. Res..

[49]  Katja Hofmann,et al.  The Malmo Platform for Artificial Intelligence Experimentation , 2016, IJCAI.

[50]  Risto Miikkulainen,et al.  Object-Model Transfer in the General Video Game Domain , 2016, AIIDE.

[51]  Julian Togelius,et al.  General Video Game AI: Competition, Challenges and Opportunities , 2016, AAAI.

[52]  Simon M. Lucas,et al.  Multi-objective tree search approaches for general video game playing , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[53]  Julian Togelius,et al.  Ieee Transactions on Computational Intelligence and Ai in Games the 2014 General Video Game Playing Competition , 2022 .

[54]  Dennis J. N. J. Soemers,et al.  Enhancements for real-time Monte-Carlo Tree Search in General Video Game Playing , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[55]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[56]  Simon M. Lucas,et al.  The N-Tuple bandit evolutionary algorithm for automatic game improvement , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[57]  Simon M. Lucas,et al.  Introducing real world physics and macro-actions to general video game ai , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[58]  J. Togelius,et al.  Cicero A mixed-initiative AI-assisted game design tool , 2017 .

[59]  Daniel A. Ashlock,et al.  General video game playing escapes the no free lunch theorem , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[60]  A. Sima Etaner-Uyar,et al.  Monte Carlo tree search with temporal-difference learning for general video game playing , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[61]  Diego Perez Liebana,et al.  Opponent models comparison for 2 players in GVGAI competitions , 2017, 2017 9th Computer Science and Electronic Engineering (CEEC).

[62]  Simon M. Lucas,et al.  Analysis of Vanilla Rolling Horizon Evolution Parameters in General Video Game Playing , 2017, EvoApplications.

[63]  Simon M. Lucas,et al.  Rolling horizon evolution enhancements in general video game playing , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[64]  Julian Togelius,et al.  Evolving Game Skill-Depth using General Video Game AI agents , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[65]  Simon M. Lucas,et al.  Population seeding techniques for Rolling Horizon Evolution in General Video Game Playing , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[66]  Julian Togelius,et al.  Multi-objective Adaptation of a Parameterized GVGAI Agent Towards Several Games , 2017, EMO.

[67]  Julian Togelius,et al.  General video game rule generation , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[68]  Julian Togelius,et al.  SeekWhence a retrospective analysis tool for general game design , 2017, FDG.

[69]  Julian Togelius,et al.  "Press Space to Fire": Automatic Video Game Tutorial Generation , 2017, AIIDE Workshops.

[70]  Annie Louis,et al.  Beyond playing to win: Diversifying heuristics for GVGAI , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[71]  Julian Togelius,et al.  Evolving Game-Specific UCB Alternatives for General Video Game Playing , 2017, EvoApplications.

[72]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[73]  Adeel Zafar,et al.  Design Patterns and General Video Game Level Generation , 2017 .

[74]  Regina Barzilay,et al.  Deep Transfer in Reinforcement Learning by Language Grounding , 2017, ArXiv.

[75]  Gabriele Kern-Isberner,et al.  An Agent-Based Learning Approach for Finding and Exploiting Heuristics in Unknown Environments , 2017, COMMONSENSE.

[76]  Simon M. Lucas,et al.  General Video Game AI: Learning from screen capture , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[77]  Ke Chen,et al.  Learning to Play General Video-Games via an Object Embedding Network , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[78]  Gillian Smith,et al.  A Design Pattern Approach for Multi-Game Level Generation , 2018, AIIDE.

[79]  Diego Pérez-Liébana,et al.  Modeling Player Experience with the N-Tuple Bandit Evolutionary Algorithm , 2018, AIIDE.

[80]  Alexander Dockhorn,et al.  Forward Model Approximation for General Video Game Learning , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[81]  Simon M. Lucas,et al.  The N-Tuple Bandit Evolutionary Algorithm for Game Agent Optimisation , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[82]  Simon M. Lucas,et al.  Using a Team of General AI Algorithms to Assist Game Design and Testing , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[83]  Simon M. Lucas,et al.  The 2016 Two-Player GVGAI Competition , 2018, IEEE Transactions on Games.

[84]  Chiara F. Sironi,et al.  Analysis of Self-Adaptive Monte Carlo Tree Search in General Video Game Playing , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[85]  Julian Togelius,et al.  Deep Reinforcement Learning for General Video Game AI , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[86]  Johannes Fürnkranz,et al.  Informed Hybrid Game Tree Search for General Video Game Playing , 2017, IEEE Transactions on Games.

[87]  Bruno Santos,et al.  An Improved Rolling Horizon Evolution Algorithm with Shift Buffer for General Game Playing , 2018, 2018 17th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames).

[88]  Julian Togelius,et al.  Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation , 2018, 1806.10729.

[89]  Julian Togelius,et al.  Deceptive Games , 2018, EvoApplications.

[90]  E. H. Santos,et al.  Redundant Action Avoidance and Non-Defeat Policy in the Monte Carlo Tree Search Algorithm for General Video Game Playing , 2018 .

[91]  Simon M. Lucas,et al.  General Win Prediction from Agent Experience , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[92]  Heder S. Bernardino,et al.  Game State Evaluation Heuristics in General Video Game Playing , 2018, 2018 17th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames).

[93]  Simon M. Lucas,et al.  Shallow Decision-Making Analysis in General Video Game Playing , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[94]  Julian Togelius,et al.  AtDELFI: automatically designing legible, full instructions for games , 2018, FDG.

[95]  Simon M. Lucas,et al.  Pac-Man Conquers Academia: Two Decades of Research Using a Classic Arcade Game , 2018, IEEE Transactions on Games.

[96]  Simon M. Lucas,et al.  Self-adaptive MCTS for General Video Game Playing , 2018, EvoApplications.

[97]  Julian Togelius,et al.  AI-Assisted Game Debugging with Cicero , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[98]  Simon M. Lucas,et al.  VERTIGØ: Visualisation of Rolling Horizon Evolutionary Algorithms in GVGAI , 2018, AIIDE.

[99]  Daniel A. Ashlock,et al.  Toward General Mathematical Game Playing Agents , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[100]  Julian Togelius,et al.  Procedural Level Generation Improves Generality of Deep Reinforcement Learning , 2018, ArXiv.

[101]  Julian Togelius,et al.  Kwiri - What, When, Where and Who: Everything You Ever Wanted to Know About Your Game But Didn't Know How to Ask , 2019, KEG@AAAI.

[102]  Simon M. Lucas,et al.  Tackling Sparse Rewards in Real-Time Games with Statistical Forward Planning Methods , 2019, AAAI.

[103]  Julian Togelius,et al.  A Continuous Information Gain Measure to Find the Most Discriminatory Problems for AI Benchmarking , 2020, 2020 IEEE Congress on Evolutionary Computation (CEC).