Multiscale Bayesian Modeling for RTS Games: An Application to StarCraft AI

This paper showcases the use of Bayesian models for real-time strategy (RTS) games AI in three distinct core components: micromanagement (units control), tactics (army moves and positions), and strategy (economy, technology, production, army types). The strength of having end-to-end probabilistic models is that distributions on specific variables can be used to interconnect different models at different levels of abstraction. We applied this modeling to StarCraft, and evaluated each model independently. Along the way, we produced and released a comprehensive data set for RTS machine learning.

[1]  Kamel Mekhnacha,et al.  Bayesian Programming , 2013 .

[2]  Santiago Ontañón,et al.  The Combinatorial Multi-Armed Bandit Problem and Its Application to Real-Time Strategy Games , 2013, AIIDE.

[3]  Pierre Bessière,et al.  A Bayesian Model for Plan Recognition in RTS Games Applied to StarCraft , 2011, AIIDE.

[4]  Gabriel Synnaeve,et al.  A Bayesian model for opening prediction in RTS games with application to StarCraft , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[5]  Curt Bererton,et al.  State Estimation for Game AI Using Particle Filters , 2004 .

[6]  Pierre Bessière,et al.  Teaching Bayesian behaviours to video game characters , 2003, Robotics Auton. Syst..

[7]  Pierre Bessière,et al.  Bayesian Modeling of a Human MMORPG Player , 2010, ArXiv.

[8]  Santiago Ontañón,et al.  Kiting in RTS Games Using Influence Maps , 2012, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[9]  Michael Mateas,et al.  A data mining approach to strategy prediction , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[10]  Gabriel Synnaeve,et al.  A Bayesian model for RTS units control applied to StarCraft , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[11]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[12]  Bhaskara Marthi,et al.  Concurrent Hierarchical Reinforcement Learning , 2005, IJCAI.

[13]  P. Bessière,et al.  Common Bayesian Models for Common Cognitive Issues , 2010, Acta biotheoretica.

[14]  Sung-Bae Cho,et al.  Replay-based strategy prediction and build order adaptation for StarCraft AI bots , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[15]  Gabriel Synnaeve,et al.  Programmation et apprentissage bayésien pour les jeux vidéo multi-joueurs, application à l'intelligence artificielle de jeux de stratégies temps-réel. (Bayesian Programming and Learning for Multi-Player Video Games, Application to RTS AI) , 2012 .

[16]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[17]  Michael Buro,et al.  Portfolio greedy search and simulation for large-scale combat in starcraft , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[18]  Luke Perkins,et al.  Terrain Analysis in Real-Time Strategy Games: An Integrated Approach to Choke Point Detection and Region Decomposition , 2010, AIIDE.

[19]  Thomas G. Dietterich,et al.  Learning Probabilistic Behavior Models in Real-Time Strategy Games , 2011, AIIDE.

[20]  ierre,et al.  Bayesian Robot Programming , 2022 .

[21]  Björn Jónsson.  Representing Uncertainty in RTS Games , 2012 .

[22]  Johannes Fürnkranz,et al.  Machines that learn to play games , 2001 .

[23]  Arnav Jhala,et al.  A Particle Model for State Estimation in Real-Time Strategy Games , 2011, AIIDE.

[24]  Nicola Beume,et al.  Towards Intelligent Team Composition and Maneuvering in Real-Time Strategy Games , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[25]  Pierre Bessière,et al.  A Dataset for StarCraft AI & an Example of Armies Clustering , 2012, ArXiv.

[26]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[27]  Dap Hartmann,et al.  MACHINES THAT LEARN TO PLAY GAMES , 2002 .

[28]  Johan Hagelbäck,et al.  Potential-field based navigation in StarCraft , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[29]  Christian Laugier,et al.  Probabilistic Reasoning and Decision Making in Sensory-Motor Systems , 2008, Springer Tracts in Advanced Robotics.

[30]  Ian D. Watson,et al.  Applying reinforcement learning to small scale combat in the real-time strategy game StarCraft:Broodwar , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[31]  Hector Muñoz-Avila,et al.  Modeling Unit Classes as Agents in Real-Time Strategy Games , 2013, AIIDE.

[32]  Pierre Bessière,et al.  Special tactics: A Bayesian approach to tactical decision-making , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[33]  Pierre Bessière,et al.  A Bayesian Tactician , 2012, CGAMES 2012.