Tactical Planning Using MCTS in the Game of StarCraft

This thesis describes how Monte-Carlo Tree Search (MCTS) can be applied to perform tactical planning for an intelligent agent playing full games of StarCraft: Brood War. StarCraft is a Real-Time Strategy game, which has a large state-space, is played in real-time, and commonly features two opposing players, capable of acting simultaneously. Using the MCTS algorithm for tactical planning is shown to increase the performance of the agent, compared to a scripted approach, when competing on a bot ladder. A combat model, based on Lanchester’s Square Law, is described, and shown to achieve another gain in performance when used in Monte-Carlo simulations as replacement for a heuristic linear model. Finally, the MAST enhancement to the Playout Policy of MCTS is described, but it is found not to have a significant impact on the agent’s performance.

[1]  Luke Perkins,et al.  Terrain Analysis in Real-Time Strategy Games: An Integrated Approach to Choke Point Detection and Region Decomposition , 2010, AIIDE.

[2]  Seth Bonder,et al.  Development of Models for Defense Systems Planning , 1970 .

[3]  Santiago Ontañón,et al.  Kiting in RTS Games Using Influence Maps , 2012, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.

[4]  Michael Buro,et al.  Portfolio greedy search and simulation for large-scale combat in starcraft , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[5]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[6]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[7]  Alan Fern,et al.  UCT for Tactical Assault Planning in Real-Time Strategy Games , 2009, IJCAI.

[8]  Jonathan Schaeffer,et al.  Monte Carlo Planning in RTS Games , 2005, CIG.

[9]  Philip M. Morse,et al.  Methods of Operations Research , 1952 .

[10]  Mark H. M. Winands,et al.  Time Management for Monte-Carlo Tree Search in Go , 2011, ACG.

[11]  Glen Robertson,et al.  A Review of Real-Time Strategy Game AI , 2014, AI Mag..

[12]  Ian Millington,et al.  Artificial Intelligence for Games , 2006, The Morgan Kaufmann series in interactive 3D technology.

[13]  Gita Reese Sukthankar,et al.  Adjutant bot: An evaluation of unit micromanagement tactics , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[14]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[15]  Stefan J. Johansson,et al.  The Rise of Potential Fields in Real Time Strategy Bots , 2008, AIIDE.

[16]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[17]  Michael Buro,et al.  Adversarial Planning Through Strategy Simulation , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[18]  F. W. Lanchester,et al.  Aircraft in Warfare: The Dawn of the Fourth Arm , 1995 .

[19]  Mark H. M. Winands,et al.  Real-Time Monte Carlo Tree Search in Ms Pac-Man , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[20]  Michael Buro.  ORTS: A Hack-Free RTS Game Environment , 2002, Computers and Games.

[21]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[22]  Stefan J. Johansson,et al.  A Multiagent Potential Field-Based Bot for Real-Time Strategy Games , 2009, Int. J. Comput. Games Technol..

[23]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[24]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[25]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[26]  Mark H. M. Winands,et al.  N-Grams and the Last-Good-Reply Policy Applied in General Game Playing , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[27]  Thomas G. Dietterich.  Adaptive computation and machine learning , 1998 .