Nested-Greedy Search for Adversarial Real-Time Games

Churchill and Buro (2013) launched a line of research through Portfolio Greedy Search (PGS), an algorithm for adversarial real-time planning that uses scripts to simplify the problem’s action space. In this paper we present a problem in PGS’s search scheme that has hitherto been overlooked. Namely, even under the strong assumption that PGS is able to evaluate all actions available to the player, PGS might fail to return the best action. We then describe an idealized algorithm that is guaranteed to return the best action and present an approximation of such algorithm, which we call NestedGreedy Search (NGS). Empirical results on μRTS show that NGS is able to outperform PGS as well as state-of-the-art methods in matches played in small to medium-sized maps. Real-time strategy (RTS) games are challenging for artificial intelligence (AI) methods. A chief difficulty faced by AI methods is the large action space encountered in such games. Churchill and Buro (2013) launched a line of research for dealing with a game’s large action space by using expertdesigned scripts. Scripts are designed to play RTS games by following simple rules such as “do not attack an enemy unit u if an ally unit will already cause enough damage to eliminate u from game”. Instead of playing the game directly with a script, Churchill and Buro used a set of scripts to define which actions should be considered during search. This way, instead of considering all legal actions during search, Churchill and Buro’s Portfolio Greedy Search (PGS) considers only the actions returned by the set of scripts. Several researchers were inspired by Churchill and Buro’s work and developed other search algorithms that use the same principle of employing a set of scripts to reduce the action space in RTS games (Justesen et al. 2014; Wang et al. 2016; Lelis 2017; Moraes and Lelis 2018). In this paper we present a problem in PGS’s search scheme that has hitherto been overlooked. Namely, even under the strong assumption that PGS is able to evaluate all actions considered by its set of scripts, the algorithm is not guaranteed to return the best available action at a given state. We call this issue the nonconvergence problem. The non-convergence problem is related to how PGS handles the responses of the player’s opponent and it might cause the algorithm to present pathological results. That is, the algorithm can produce worse results Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. if allowed more computation time. We show empirically in the context of μRTS, a minimalist RTS game for research purposes, that PGS’s pathology is very common in practice. In this paper we also present a search algorithm called Nested-Greedy Search (NGS) to overcome PGS’s nonconvergence problem. NGS is similar to PGS, with the only difference being how the algorithm handles the enemy responses during search. In contrast with PGS, NGS approximates how the opponent could best respond to different actions of the player and returns the action that yields the largest payoff for the player, assuming the opponent will play an approximated best response. We evaluated NGS in μRTS matches. Our empirical results show that NGS is able to outperform not only PGS, but all state-of-the-art methods tested in matches played in small to medium-sized maps. In addition to presenting the non-convergence problem as well as a search algorithm to overcome the problem, another contribution of this work is to show that PGS and NGS can be used to play entire RTS matches. This is important because PGS was developed to control units in combat scenarios that arise in RTS games, and not to play entire RTS matches, which requires one to deal with the economical side of the game in addition to the military side of the game. Our work suggests that other researchers should consider PGS, NGS, and other algorithms derived from PGS as competing methods for their planning systems for RTS games.