A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft

This paper presents an overview of the existing work on AI for real-time strategy (RTS) games. Specifically, we focus on the work around the game StarCraft, which has emerged in the past few years as the unified test bed for this research. We describe the specific AI challenges posed by RTS games, and overview the solutions that have been explored to address them. Additionally, we also present a summary of the results of the recent StarCraft AI competitions, describing the architectures used by the participants. Finally, we conclude with a discussion emphasizing which problems in the context of RTS game AI have been solved, and which remain open.

[1]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[2]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[3]  Craig W. Reynolds.  Steering Behaviors For Autonomous Characters , 1999 .

[4]  Ioannis P. Vlahavas,et al.  Heuristic Planning with Resources , 2000, ECAI.

[5]  Gerald Tesauro,et al.  Comparison training of chess evaluation functions , 2001 .

[6]  Kenneth D. Forbus,et al.  How qualitative spatial reasoning can improve strategy game AIs , 2002, IEEE Intelligent Systems.

[7]  Maxim Likhachev,et al.  D*lite , 2002, AAAI/IAAI.

[8]  Michael Buro,et al.  Real-Time Strategy Games: A New AI Research Challenge , 2003, IJCAI.

[9]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[10]  Marc J. V. Ponsen,et al.  Improving Adaptive Game Ai with Evolutionary Learning , 2004 .

[11]  Bhaskara Marthi,et al.  Concurrent Hierarchical Reinforcement Learning , 2005, IJCAI.

[12]  Hector Muñoz-Avila,et al.  Hierarchical Plan Representations for Encoding Strategic Game AI , 2005, AIIDE.

[13]  Jonathan Schaeffer,et al.  Monte Carlo Planning in RTS Games , 2005, CIG.

[14]  David W. Aha,et al.  Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game , 2005, Künstliche Intell..

[15]  Vincent Corruble,et al.  Designing a Reinforcement Learning-based Adaptive AI for Large-Scale Strategy Games , 2006, AIIDE.

[16]  Adrien Treuille,et al.  Continuum crowds , 2006, SIGGRAPH 2006.

[17]  Michael Buro,et al.  Efficient Triangulation-Based Pathfinding , 2006, AAAI.

[18]  Sushil J. Louis,et al.  Co-evolving real-time strategy game players , 2007 .

[19]  John E. Laird,et al.  SORTS: A Human-Level Approach to Real-Time Strategy AI , 2007, AIIDE.

[20]  Pieter Spronck,et al.  Opponent Modeling in Real-Time Strategy Games , 2007, GAMEON.

[21]  Ashwin Ram,et al.  Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL , 2007, IJCAI.

[22]  Stefan J. Johansson,et al.  Dealing with fog of war in a Real Time Strategy game environment , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[23]  D. Hunter Hale,et al.  Automatically-generated Convex Region Decomposition for Real-time Spatial Agent Navigation in Virtual Worlds , 2021, AIIDE.

[24]  Santiago Ontañón,et al.  Situation Assessment for Plan Retrieval in Real-Time Strategy Games , 2008, ECCBR.

[25]  Liang Liu,et al.  Regional Cooperative Multi-agent Q-learning Based on Potential Field , 2008, 2008 Fourth International Conference on Natural Computation.

[26]  David W. Aha,et al.  Learning Continuous Action Models in a Real-Time Strategy Environment , 2008, FLAIRS.

[27]  Vadim Bulitko,et al.  An evaluation of models for predicting opponent positions in first-person shooter video games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[28]  Nicola Beume,et al.  Intelligent moving of groups in real-time strategy games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[29]  Santiago Ontañón,et al.  Learning from Demonstration and Case-Based Planning for Real-Time Strategy Games , 2008, Soft Computing Applications in Industry.

[30]  Chuen-Tsai Sun,et al.  Building a player strategy model by analyzing replays of real-time strategy games , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[31]  Alan Fern,et al.  UCT for Tactical Assault Planning in Real-Time Strategy Games , 2009, IJCAI.

[32]  Stefan J. Johansson,et al.  A Multiagent Potential Field-Based Bot for Real-Time Strategy Games , 2009, Int. J. Comput. Games Technol..

[33]  Sushil J. Louis,et al.  Evolving coordinated spatial tactics for autonomous entities using influence maps , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[34]  Michael Mateas,et al.  A data mining approach to strategy prediction , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[35]  Robert P. Goldman,et al.  A probabilistic plan recognition algorithm based on plan tree grammars , 2009, Artif. Intell..

[36]  Arnav Jhala,et al.  Applying Goal-Driven Autonomy to StarCraft , 2010, AIIDE.

[37]  Froduald Kabanza,et al.  Opponent Behaviour Recognition for Real-Time Strategy Games , 2010, Plan, Activity, and Intent Recognition.

[38]  Luke Perkins,et al.  Terrain Analysis in Real-Time Strategy Games: An Integrated Approach to Choke Point Detection and Region Decomposition , 2010, AIIDE.

[39]  Nicola Beume,et al.  Towards Intelligent Team Composition and Maneuvering in Real-Time Strategy Games , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[40]  Arnav Jhala,et al.  Reactive planning idioms for multi-scale game AI , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[41]  Sushil J. Louis,et al.  Using co-evolved RTS opponents to teach spatial tactics , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[42]  Santiago Ontañón,et al.  ON‐LINE CASE‐BASED PLANNING , 2010, Comput. Intell..

[43]  Michael Buro,et al.  Build Order Optimization in StarCraft , 2011, AIIDE.

[44]  Arnav Jhala,et al.  A Particle Model for State Estimation in Real-Time Strategy Games , 2011, AIIDE.

[45]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[46]  Thomas G. Dietterich,et al.  Learning Probabilistic Behavior Models in Real-Time Strategy Games , 2011, AIIDE.

[47]  Pierre Bessière,et al.  A Bayesian Model for Plan Recognition in RTS Games Applied to StarCraft , 2011, AIIDE.

[48]  Gabriel Synnaeve,et al.  A Bayesian model for RTS units control applied to StarCraft , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[49]  Regina Barzilay,et al.  Learning to Win by Reading Manuals in a Monte-Carlo Framework , 2011, ACL.

[50]  Arnav Jhala,et al.  Building Human-Level AI for Real-Time Strategy Games , 2011, AAAI Fall Symposium: Advances in Cognitive Systems.

[51]  Gabriel Synnaeve,et al.  A Bayesian model for opening prediction in RTS games with application to StarCraft , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[52]  Leonardo Garrido,et al.  Fuzzy Case-Based Reasoning for Managing Strategic and Tactical Reasoning in StarCraft , 2011, MICAI.

[53]  D. Aha,et al.  Case-Based Learning in Goal-Driven Autonomy Agents for Real-Time Strategy Combat Tasks , 2011 .

[54]  Ian D. Watson,et al.  Applying reinforcement learning to small scale combat in the real-time strategy game StarCraft:Broodwar , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[55]  Michael Buro,et al.  Fast Heuristic Search for RTS Game Combat Scenarios , 2012, AIIDE.

[56]  Johan Hagelbäck,et al.  Potential-field based navigation in StarCraft , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[57]  Nick Hawes,et al.  Evolutionary Learning of Goal Priorities in a Real-Time Strategy Game , 2012, AIIDE.

[58]  Nathan R. Sturtevant,et al.  Benchmarks for Grid-Based Pathfinding , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[59]  Pierre Bessière,et al.  Special tactics: A Bayesian approach to tactical decision-making , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[60]  Michael Buro,et al.  Real-Time Strategy Game Competitions , 2012, AI Mag..

[61]  Tom Chothia,et al.  SCAIL: An integrated Starcraft AI system , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[62]  Pierre Bessière,et al.  A Dataset for StarCraft AI & an Example of Armies Clustering , 2012, ArXiv.

[63]  Santiago Ontañón,et al.  Kiting in RTS Games Using Influence Maps , 2012, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[64]  Héctor Muñoz-Avila,et al.  CLASSQ-L: A Q-Learning Algorithm for Adversarial Real-Time Strategy Games , 2012, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[65]  Wentong Cai,et al.  Simulation-based optimization of StarCraft tactical AI through evolutionary computation , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[66]  Michal Certický,et al.  Implementing a Wall-In Building Placement in StarCraft with Declarative Programming , 2013, ArXiv.

[67]  Santiago Ontañón,et al.  The Combinatorial Multi-Armed Bandit Problem and Its Application to Real-Time Strategy Games , 2013, AIIDE.

[68]  C. Miles.  Co-evolving Real-Time Strategy Game Playing Influence Map Trees With Genetic Algorithms , 2022 .

[69]  Martin Certický,et al.  Case-Based Reasoning for Army Compositions in Real-Time Strategy Games , 2022 .