Discovering optimal strategy in tactical combat scenarios through the evolution of behaviour trees

In this paper we address the problem of automatically discovering optimal tactics in a combat scenario in which two opposing sides control a number of fighting units. Our approach is based on the evolution of behaviour trees, combined with simulation-based evaluation of solutions to drive the evolution. Our behaviour trees use a small set of possible actions that can be assigned to a combat unit, along with standard behaviour tree constructs and a novel approach for selecting which action from the tree is performed. A set of test scenarios was designed for which an optimal strategy is known from the literature. These scenarios were used to explore and evaluate our approach. The results indicate that it is possible, from the small set of possible unit actions, for a complex strategy to emerge through evolution. Combat units with different capabilities were observed exhibiting coordinated team work and exploiting aspects of the environment.

[1]  Santiago Ontañón,et al.  The Combinatorial Multi-Armed Bandit Problem and Its Application to Real-Time Strategy Games , 2013, AIIDE.

[2]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[3]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[4]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[5]  Glen Robertson,et al.  Building behavior trees from observations in real-time strategy games , 2015, 2015 International Symposium on Innovations in Intelligent SysTems and Applications (INISTA).

[6]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[7]  Sachin Modgil,et al.  Artificial intelligence for decision support systems in the field of operations research: review and future scope of research , 2021, Annals of Operations Research.

[8]  Glen Robertson,et al.  A Review of Real-Time Strategy Game AI , 2014, AI Mag..

[9]  Gilbert Laporte,et al.  Metaheuristics: A bibliography , 1996, Ann. Oper. Res..

[10]  Siming Liu,et al.  Neuroevolution for RTS Micro , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[11]  Julian Togelius,et al.  Super mario evolution , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[12]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[13]  Anthony Brabazon,et al.  Reactiveness and navigation in computer games: Different needs, different approaches , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[14]  Jonatan Wilhelm Hoff,et al.  Evolving Behaviour Trees: - Automatic Generation of AI Opponents for Real-Time Strategy Games , 2016 .

[15]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[16]  E. James Whitehead,et al.  Design patterns in FPS levels , 2010, FDG.

[17]  Rikke Amilde Løvlid,et al.  Automating Behaviour Tree Generation for Simulating Troop Movements (Poster) , 2018, 2018 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA).

[18]  Simon Colton,et al.  Evolving Behaviour Trees for the Commercial Game DEFCON , 2010, EvoApplications.

[19]  Michael Papasimeon,et al.  A simheuristic approach for evolving agent behaviour in the exploration for novel combat tactics , 2019, Operations Research Perspectives.

[20]  Angel A. Juan,et al.  A review of simheuristics: Extending metaheuristics to deal with stochastic combinatorial optimization problems , 2015 .

[21]  Chiou Peng Lam,et al.  Evolving behaviour trees for automated discovery of novel combat strategy in real-time strategy wargames , 2019 .

[22]  Angel A. Juan,et al.  A review of the role of heuristics in stochastic optimisation: from metaheuristics to learnheuristics , 2021, Annals of Operations Research.