Portfolio Search and Optimization for General Strategy Game-Playing

Portfolio methods represent a simple but efficient type of action abstraction which has shown to improve the performance of search-based agents in a range of strategy games. We first review existing portfolio techniques and propose a new algorithm for optimization and action-selection based on the Rolling Horizon Evolutionary Algorithm. Moreover, a series of variants are developed to solve problems in different aspects. We further analyze the performance of discussed agents in a general strategy game-playing task. For this purpose, we run experiments on three different game-modes of the Stratega framework. For the optimization of the agents' parameters and portfolio sets we study the use of the N-tuple Bandit Evolutionary Algorithm. The resulting portfolio sets suggest a high diversity in play-styles while being able to consistently beat the sample agents. An analysis of the agents' performance shows that the proposed algorithm generalizes well to all game-modes and is able to outperform other portfolio methods.

[1]  Diego Perez Liebana,et al.  The Design Of "Stratega": A General Strategy Games Framework , 2020, ArXiv.

[2]  Levi Lelis,et al.  Planning Algorithms for Zero-Sum Games with Exponential Action Spaces: A Unifying Perspective , 2020, IJCAI.

[3]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[4]  Julian Togelius,et al.  General Video Game Artificial Intelligence , 2019, Synthesis Lectures on Games and Computational Intelligence.

[5]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[6]  Levi Lelis,et al.  Nested-Greedy Search for Adversarial Real-Time Games , 2018, AIIDE.

[7]  Julian Togelius,et al.  Playing Multiaction Adversarial Games: Online Evolutionary Planning Versus Tree Search , 2018, IEEE Transactions on Games.

[8]  Julian Togelius,et al.  General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms , 2018, IEEE Transactions on Games.

[9]  Simon M. Lucas,et al.  The N-Tuple Bandit Evolutionary Algorithm for Game Agent Optimisation , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[10]  Simon M. Lucas,et al.  Analysis of Vanilla Rolling Horizon Evolution Parameters in General Video Game Playing , 2017, EvoApplications.

[11]  Julian Togelius,et al.  Online Evolution for Multi-action Adversarial Games , 2016, EvoApplications.

[12]  Jean-Baptiste Mouret,et al.  Illuminating search spaces by mapping elites , 2015, ArXiv.

[13]  Michael Buro,et al.  Portfolio greedy search and simulation for large-scale combat in starcraft , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[14]  Simon M. Lucas,et al.  Rolling horizon evolution versus tree search for navigation in single-player real-time games , 2013, GECCO '13.

[15]  Michael R. Genesereth,et al.  General Game Playing: Overview of the AAAI Competition , 2005, AI Mag..

[16]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[17]  Alexander Dockhorn.  Prediction-based search for autonomous game-playing , 2020 .

[18]  Diego Perez Liebana,et al.  STRATEGA: A General Strategy Games Framework , 2020, AIIDE Workshops.

[19]  W. Marsden.  I and J , 2012 .

[20]  I. Miyazaki,et al.  AND T , 2022 .