A Bayesian model for RTS units control applied to StarCraft

In real-time strategy games (RTS), the player must reason about high-level strategy and planning while having effective tactics and even individual units micro-management. Enabling an artificial agent to deal with such a task entails breaking down the complexity of this environment. For that, we propose to control units locally in the Bayesian sensory motor robot fashion, with higher level orders integrated as perceptions. As complete inference encompassing global strategy down to individual unit needs is intractable, we embrace incompleteness through a hierarchical model able to deal with uncertainty. We developed and applied our approach on a StarCraft1 AI.

[1]  David W. Aha,et al.  Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game , 2005, Künstliche Intell..

[2]  ierre,et al.  Bayesian Robot Programming , 2022 .

[3]  Alan Fern,et al.  UCT for Tactical Assault Planning in Real-Time Strategy Games , 2009, IJCAI.

[4]  Santiago Ontañón,et al.  Case-Based Planning and Execution for Real-Time Strategy Games , 2007, ICCBR.

[5]  Ashwin Ram,et al.  Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL , 2007, IJCAI.

[6]  Arnav Jhala,et al.  Reactive planning idioms for multi-scale game AI , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[7]  Bhaskara Marthi,et al.  Concurrent Hierarchical Reinforcement Learning , 2005, IJCAI.

[8]  Jonathan Schaeffer,et al.  Monte Carlo Planning in RTS Games , 2005, CIG.

[9]  Pierre Bessière,et al.  A Survey of Probabilistic Models Using the Bayesian Programming Methodology as a Unifying Framework , 2003 .

[10]  Vincent Corruble,et al.  Designing a Reinforcement Learning-based Adaptive AI for Large-Scale Strategy Games , 2006, AIIDE.

[11]  John E. Laird,et al.  SORTS: A Human-Level Approach to Real-Time Strategy AI , 2007, AIIDE.

[12]  Marc J. V. Ponsen,et al.  Improving Adaptive Game Ai with Evolutionary Learning , 2004 .

[13]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[14]  Christian Laugier,et al.  Probabilistic Reasoning and Decision Making in Sensory-Motor Systems , 2008, Springer Tracts in Advanced Robotics.

[15]  P. Bessière,et al.  Common Bayesian Models for Common Cognitive Issues , 2010, Acta biotheoretica.

[16]  Tutorial-Bob Reynolds,et al.  2011 IEEE Conference on Computational Intelligence and Games , 2011 .

[17]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[18]  Stefan J. Johansson,et al.  A Multiagent Potential Field-Based Bot for Real-Time Strategy Games , 2009, Int. J. Comput. Games Technol..

[19]  Pierre Bessière,et al.  Teaching Bayesian behaviours to video game characters , 2003, Robotics Auton. Syst..

[20]  David W. Aha,et al.  Learning Continuous Action Models in a Real-Time Strategy Environment , 2008, FLAIRS.

[21]  Lihong Li,et al.  A Bayesian Sampling Approach to Exploration in Reinforcement Learning , 2009, UAI.

[22]  Nicola Beume,et al.  Towards Intelligent Team Composition and Maneuvering in Real-Time Strategy Games , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[23]  Duane Szafron,et al.  An Architecture for Game Behavior AI: Behavior Multi-Queues , 2009, AIIDE.