Evolving effective micro behaviors in RTS game

We investigate using genetic algorithms to generate high quality micro management in combat scenarios for real-time strategy games. Macro and micro management are two key aspects of real-time strategy games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields to generate micro management positioning and movement tactics. Micro behaviors are compactly encoded into fourteen parameters and we use genetic algorithms to search for effective micro management tactics for the given units. We tested the performance of our ECSLBot (the evolved player), obtained in this way against the default StarCraft AI, and two other state of the art bots, UAlbertaBot and Nova on several skirmish scenarios. The results show that the ECSLBot tuned by genetic algorithms outperforms the UAlbertaBot and Nova in kiting efficiency, target selection, and knowing when to flee to survive. We believe our approach is easy to extend to other types of units and can be easily adopted by other AI bots.

[1]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[2]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1998 .

[3]  Sung-Bae Cho,et al.  Evolving neural NPCs with layered influence map in the real-time simulation game ‘Conqueror’ , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[4]  Santiago Ontañón,et al.  Kiting in RTS Games Using Influence Maps , 2012, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[5]  Yoram Koren,et al.  The vector field histogram-fast obstacle avoidance for mobile robots , 1991, IEEE Trans. Robotics Autom..

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  Keith D. Rogers,et al.  A Micromanagement Task Allocation System for Real-Time Strategy Games , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[8]  Stefan J. Johansson,et al.  Using multi-agent potential fields in real-time strategy games , 2008, AAMAS.

[9]  Pieter Spronck,et al.  Adaptive Spatial Reasoning for Turn-based Strategy Games , 2008, AIIDE.

[10]  Sushil J. Louis,et al.  Co-Evolving Influence Map Tree Based Strategy Game Players , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[11]  Sushil J. Louis,et al.  Coevolving influence maps for spatial team tactics in a RTS game , 2010, GECCO '10.

[12]  Ugur Kuter,et al.  Adversarial Planning for Multi-Agent Pursuit-Evasion Games in Partially Observable Euclidean Space , 2012 .

[13]  Nicola Beume,et al.  Towards Intelligent Team Composition and Maneuvering in Real-Time Strategy Games , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[14]  Michael Buro,et al.  Build Order Optimization in StarCraft , 2011, AIIDE.

[15]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[16]  Xiaoming Hu,et al.  Formation constrained multi-agent control , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[17]  Nicola Beume,et al.  Intelligent moving of groups in real-time strategy games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[18]  Michael Buro,et al.  Real-Time Strategy Games: A New AI Research Challenge , 2003, IJCAI.

[19]  Martin Johansen Gunnerud.  A CBR/RL system for learning micromanagement in real-time strategy games , 2009 .

[20]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[21]  Monica N. Nicolescu,et al.  Comparing heuristic search methods for finding effective group behaviors in RTS game , 2013, 2013 IEEE Congress on Evolutionary Computation.

[22]  Janet Wiles,et al.  Combining Influence Maps and Cellular Automata for Reactive Game Agents , 2005, IDEAL.

[23]  Stefan J. Johansson,et al.  The Rise of Potential Fields in Real Time Strategy Bots , 2008, AIIDE.

[24]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[25]  Monica N. Nicolescu,et al.  Using CIGAR for finding effective group behaviors in RTS game , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[26]  Ian D. Watson,et al.  Applying reinforcement learning to small scale combat in the real-time strategy game StarCraft:Broodwar , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).