Does it matter how well I know what you’re thinking? Opponent Modelling in an RTS game

Opponent Modelling tries to predict the future actions of opponents, and is required to perform well in multiplayer games. There is a deep literature on learning an opponent model, but much less on how accurate such models must be to be useful. We investigate the sensitivity of Monte Carlo Tree Search (MCTS) and a Rolling Horizon Evolutionary Algorithm (RHEA) to the accuracy of their modelling of the opponent in a simple Real-Time Strategy game. We find that in this domain RHEA is much more sensitive to the accuracy of an opponent model than MCTS. MCTS generally does better even with an inaccurate model, while this will degrade RHEA’s performance. We show that faced with an unknown opponent and a low computational budget it is better not to use any explicit model with RHEA, and to model the opponent’s actions within the tree as part of the MCTS algorithm.

[1]  Michael H. Bowling,et al.  Finding Optimal Abstract Strategies in Extensive-Form Games , 2012, AAAI.

[2]  Michael Buro,et al.  Portfolio greedy search and simulation for large-scale combat in starcraft , 2013, 2013 IEEE Conference on Computational Inteligence in Games (CIG).

[3]  Simon M. Lucas,et al.  Evaluating and modelling Hanabi-playing agents , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[4]  Joseph H. Engel,et al.  A Verification of Lanchester's Law , 1954, Oper. Res..

[5]  Levente Kocsis,et al.  Transpositions and move groups in Monte Carlo tree search , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[6]  Robert L. Helmbold,et al.  Validating Lanchester's square law and other attrition models , 1995 .

[7]  Mario A. Nascimento,et al.  Action Abstractions for Combinatorial Multi-Armed Bandit Tree Search , 2018, AIIDE.

[8]  Michael Buro,et al.  Learning Policies from Human Data for Skat , 2019, 2019 IEEE Conference on Games (CoG).

[9]  Simon M. Lucas,et al.  Rolling horizon evolution enhancements in general video game playing , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[10]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[11]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[12]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[13]  Peter Stone,et al.  Autonomous agents modelling other agents: A comprehensive survey and open problems , 2017, Artif. Intell..

[14]  H. J. van den Herik,et al.  Opponent Modelling and Commercial Games , 2005 .

[15]  D. Stahl,et al.  On Players' Models of Other Players: Theory and Experimental Evidence , 1995 .

[16]  Simon M. Lucas,et al.  Learning Local Forward Models on Unforgiving Games , 2019, 2019 IEEE Conference on Games (CoG).

[17]  Colin Camerer,et al.  A Cognitive Hierarchy Model of Games , 2004 .

[18]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[19]  Simon M. Lucas,et al.  Open Loop Search for General Video Game Playing , 2015, GECCO.

[20]  Joshua B. Tenenbaum,et al.  Theory of Minds: Understanding Behavior in Groups Through Inverse Planning , 2019, AAAI.

[21]  Javier Peña,et al.  Gradient-Based Algorithms for Finding Nash Equilibria in Extensive Form Games , 2007, WINE.

[22]  Simon M. Lucas,et al.  The N-Tuple Bandit Evolutionary Algorithm for Game Agent Optimisation , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[23]  Jos W. H. M. Uiterwijk,et al.  Monte-Carlo tree search in production management problems , 2006 .

[24]  Nathan R. Sturtevant,et al.  Robust game play against unknown opponents , 2006, AAMAS '06.

[25]  David Carmel,et al.  Incorporating Opponent Models into Adversary Search , 1996, AAAI/IAAI, Vol. 1.

[26]  Santiago Ontañón,et al.  Combinatorial Multi-armed Bandits for Real-Time Strategy Games , 2017, J. Artif. Intell. Res..

[27]  Santiago Ontañón,et al.  Adversarial Hierarchical-Task Network Planning for Complex Real-Time Games , 2015, IJCAI.

[28]  Simon M. Lucas,et al.  A Local Approach to Forward Model Learning: Results on the Game of Life Game , 2019, 2019 IEEE Conference on Games (CoG).

[29]  Simon M. Lucas,et al.  Rolling horizon evolution versus tree search for navigation in single-player real-time games , 2013, GECCO '13.

[30]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[31]  Yoshimasa Tsuruoka,et al.  Building a computer Mahjong player based on Monte Carlo simulation and opponent models , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[32]  Simon M. Lucas.  Game AI Research with Fast Planet Wars Variants , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[33]  Edmund H. Durfee,et al.  A Rigorous, Operational Formalization of Recursive Modeling , 1995, ICMAS.

[34]  Simon M. Lucas,et al.  Rolling Horizon Coevolutionary planning for two-player video games , 2016, 2016 8th Computer Science and Electronic Engineering (CEEC).

[35]  Demis Hassabis,et al.  Mastering Atari, Go, chess and shogi by planning with a learned model , 2019, Nature.

[36]  Simon M. Lucas,et al.  Efficient Evolutionary Methods for Game Agent Optimisation: Model-Based is Best , 2019, ArXiv.