A Review of Real-Time Strategy Game AI

This literature review covers AI techniques used for real-time strategy video games, focusing specifically on StarCraft. It finds that the main areas of current academic research are in tactical and strategic decision-making, plan recognition, and learning, and it outlines the research contributions in each of these areas. The paper then contrasts the use of game AI in academia and industry, finding the academic research heavily focused on creating game-winning agents, while the indus- try aims to maximise player enjoyment. It finds the industry adoption of academic research is low because it is either in- applicable or too time-consuming and risky to implement in a new game, which highlights an area for potential investi- gation: bridging the gap between academia and industry. Fi- nally, the areas of spatial reasoning, multi-scale AI, and co- operation are found to require future work, and standardised evaluation methods are proposed to produce comparable re- sults between studies.

[1]  V. Rich.  Personal communication , 1989, Nature.

[2]  Steve Rabin.  The illusion of intelligence , 1991, The Lancet.

[3]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[4]  Hiroaki Kitano,et al.  The RoboCup Synthetic Agent Challenge 97 , 1997, IJCAI.

[5]  Ian Lane Davis,et al.  Strategies for Strategy Game AI , 1999 .

[6]  Einar M. Rønquist,et al.  Foreword , 1999, Biological Psychiatry.

[7]  Jonathan Schaeffer,et al.  A Gamut of Games , 2001, AI Mag..

[8]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[9]  Andrew Stern,et al.  A Behavior Language for Story-Based Believable Agents , 2002, IEEE Intell. Syst..

[10]  Michael Buro,et al.  RTS Games and Real-Time AI Research , 2003 .

[11]  Michael Buro,et al.  Real-Time Strategy Games: A New AI Research Challenge , 2003, IJCAI.

[12]  D. Aha,et al.  On the Role of Explanation for Hierarchical Case-Based Planning in Real-Time Strategy Games , 2004 .

[13]  Ruck Thawonmas,et al.  CASE-BASED PLAN RECOGNITION FOR REAL-TIME STRATEGY GAMES , 2004 .

[14]  Mat Buckland,et al.  Programming Game AI by Example , 2004 .

[15]  Michael Buro,et al.  Call for AI Research in RTS Games , 2004 .

[16]  David W. Aha,et al.  Integrating Learning in Interactive Gaming Simulators , 2004 .

[17]  Bhaskara Marthi,et al.  Concurrent Hierarchical Reinforcement Learning , 2005, IJCAI.

[18]  David W. Aha,et al.  Automatically Acquiring Domain Knowledge For Adaptive Game AI Using Evolutionary Learning , 2005, AAAI.

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  Marco Antonio Gómez-Martín,et al.  A CBR Module for a Strategy Videogame , 2005, ICCBR Workshops.

[21]  Jonathan Schaeffer,et al.  Monte Carlo Planning in RTS Games , 2005, CIG.

[22]  David W. Aha,et al.  Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game , 2005, Künstliche Intell..

[23]  David W. Aha,et al.  Automatically Generating Game Tactics through Evolutionary Learning , 2006, AI Mag..

[24]  Pat Langley,et al.  Learning hierarchical task networks by observation , 2006, ICML.

[25]  John E. Laird,et al.  SORTS: A Human-Level Approach to Real-Time Strategy AI , 2007, AIIDE.

[26]  Alan Fern,et al.  Online Planning for Resource Production in Real-Time Strategy Games , 2007, ICAPS.

[27]  D.H. Grollman,et al.  Learning robot soccer skills from demonstration , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[28]  Ashwin Ram,et al.  Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL , 2007, IJCAI.

[29]  Santiago Ontañón,et al.  Case-Based Planning and Execution for Real-Time Strategy Games , 2007, ICCBR.

[30]  Michael Buro,et al.  Adversarial Planning Through Strategy Simulation , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[31]  Jeff Orkin,et al.  Applying Goal-Oriented Action Planning to Games , 2008 .

[32]  Santiago Ontañón,et al.  Situation Assessment for Plan Retrieval in Real-Time Strategy Games , 2008, ECCBR.

[33]  David W. Aha,et al.  Learning Continuous Action Models in a Real-Time Strategy Environment , 2008, FLAIRS.

[34]  Stefan J. Johansson,et al.  The Rise of Potential Fields in Real Time Strategy Bots , 2008, AIIDE.

[35]  Chuen-Tsai Sun,et al.  Building a player strategy model by analyzing replays of real-time strategy games , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[36]  Simon Colton,et al.  Combining AI Methods for Learning Bots in a Real-Time Strategy Game , 2009, Int. J. Comput. Games Technol..

[37]  Alan Fern,et al.  UCT for Tactical Assault Planning in Real-Time Strategy Games , 2009, IJCAI.

[38]  A. Ram,et al.  Authoring Behaviors for Games using Learning from Demonstration , 2009 .

[39]  Agnar Aamodt.  Case-based reasoning for improved micromanagement in Real-time strategy games , 2009 .

[40]  Michael W. Floyd,et al.  Comparison of Classifiers for use in a Learning by Demonstration System for a Situated Agent , 2009 .

[41]  Michael Mateas,et al.  A data mining approach to strategy prediction , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[42]  Stefan J. Johansson,et al.  Measuring player experience on runtime dynamic difficulty scaling in an RTS game , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[43]  Arnav Jhala,et al.  Applying Goal-Driven Autonomy to StarCraft , 2010, AIIDE.

[44]  Froduald Kabanza,et al.  Opponent Behaviour Recognition for Real-Time Strategy Games , 2010, Plan, Activity, and Intent Recognition.

[45]  Luke Perkins,et al.  Terrain Analysis in Real-Time Strategy Games: An Integrated Approach to Choke Point Detection and Region Decomposition , 2010, AIIDE.

[46]  Santiago Ontañón,et al.  Using Automated Replay Annotation for Case-Based Planning in Games , 2010 .

[47]  Arnav Jhala,et al.  Reactive planning idioms for multi-scale game AI , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[48]  David W. Aha,et al.  Goal-Driven Autonomy in a Navy Strategy Simulation , 2010, AAAI.

[49]  Thomas G. Dietterich,et al.  Reinforcement Learning Via Practice and Critique Advice , 2010, AAAI.

[50]  Michael W. Floyd,et al.  Toward a Domain-independent Case-based Reasoning Approach for Imitation : Three Case Studies in Gaming , 2010 .

[51]  Michael Buro,et al.  Build Order Optimization in StarCraft , 2011, AIIDE.

[52]  Arnav Jhala,et al.  A Particle Model for State Estimation in Real-Time Strategy Games , 2011, AIIDE.

[53]  Thomas G. Dietterich,et al.  Learning Probabilistic Behavior Models in Real-Time Strategy Games , 2011, AIIDE.

[54]  Pedro Pablo Gómez-Martín,et al.  Combining Expert Knowledge and Learning from Demonstration in Real-Time Strategy Games , 2011, ICCBR.

[55]  Marco Wiering,et al.  Connectionist reinforcement learning for intelligent unit micro management in StarCraft , 2011, The 2011 International Joint Conference on Neural Networks.

[56]  Pierre Bessière,et al.  A Bayesian Model for Plan Recognition in RTS Games Applied to StarCraft , 2011, AIIDE.

[57]  David W. Aha,et al.  Integrated Learning for Goal-Driven Autonomy , 2011, IJCAI.

[58]  Gabriel Synnaeve,et al.  A Bayesian model for RTS units control applied to StarCraft , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[59]  Santiago Ontañón.  Towards a Unified Framework for Learning from Observation , 2011 .

[60]  Babak Esfandiari,et al.  Learning state-based behaviour using temporally related cases , 2011 .

[61]  Arnav Jhala,et al.  Building Human-Level AI for Real-Time Strategy Games , 2011, AAAI Fall Symposium: Advances in Cognitive Systems.

[62]  Pieter Spronck,et al.  A CBR Inspired Approach to Rapid and Reliable Adaption of Video Game Al , 2011 .

[63]  Leonardo Garrido,et al.  Fuzzy Case-Based Reasoning for Managing Strategic and Tactical Reasoning in StarCraft , 2011, MICAI.

[64]  Babak Esfandiari,et al.  A Case-Based Reasoning Framework for Developing Agents Using Learning by Observation , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[65]  Michael Buro,et al.  Fast Heuristic Search for RTS Game Combat Scenarios , 2012, AIIDE.

[66]  Arnav Jhala,et al.  Learning from Demonstration for Goal-Driven Autonomy , 2012, AAAI.

[67]  Santiago Ontañón,et al.  Case Acquisition Strategies for Case-Based Reasoning in Real-Time Strategy Games , 2012, FLAIRS.

[68]  Gabriel Synnaeve,et al.  A Dataset for StarCraft AI and an Example of Armies Clustering , 2012 .

[69]  Matteus Magnusson,et al.  A Communicating and Controllable Teammate Bot for RTS Games , 2012 .

[70]  Michael Buro,et al.  Real-Time Strategy Game Competitions , 2012, AI Mag..

[71]  Santiago Ontañón,et al.  Kiting in RTS Games Using Influence Maps , 2012, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[72]  Thomas G. Dietterich,et al.  Inferring Strategies from Limited Reconnaissance in Real-time Strategy Games , 2012, UAI.

[73]  Daniela Zaharie,et al.  Neuroevolution based multi-agent system for micromanagement in real-time strategy games , 2012, BCI '12.

[74]  Michael Buro,et al.  Incorporating Search Algorithms into RTS Game Agents , 2012 .

[75]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.