Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using Evolutionary Algorithms

This paper investigates the performance and the results of an evolutionary algorithm (EA) specifically designed for evolving the decision engine of a program (which, in this context, is called bot) that plays Planet Wars. This game, which was chosen for the Google Artificial Intelligence Challenge in 2010, requires the bot to deal with multiple target planets, while achieving a certain degree of adaptability in order to defeat different opponents in different scenarios. The decision engine of the bot is initially based on a set of rules that have been defined after an empirical study, and a genetic algorithm (GA) is used for tuning the set of constants, weights and probabilities that those rules include, and therefore, the general behaviour of the bot. Then, the bot is supplied with the evolved decision engine and the results obtained when competing with other bots (a bot offered by Google as a sparring partner, and a scripted bot with a pre-established behaviour) are thoroughly analysed. The evaluation of the candidate solutions is based on the result of non-deterministic battles (and environmental interactions) against other bots, whose outcome depends on random draws as well as on the opponents’ actions. Therefore, the proposed GA is dealing with a noisy fitness function. After analysing the effects of the noisy fitness, we conclude that tackling randomness via repeated combats and reevaluations reduces this effect and makes the GA a highly valuable approach for solving this problem.

[1]  Eric O. Postma,et al.  Improved opponent intelligence trough offline learning , 2003, Int. J. Intell. Games Simul..

[2]  Juan Julián Merelo Guervós,et al.  Dealing with Noisy Fitness in the Design of a RTS Game Bot , 2012, EvoApplications.

[3]  Simon M. Lucas.  Computational intelligence and games: Challenges and opportunities , 2008, Int. J. Autom. Comput..

[4]  Santiago Ontañón,et al.  Case-Based Planning and Execution for Real-Time Strategy Games , 2007, ICCBR.

[5]  John E. Laird,et al.  Using a Computer Game to Develop Advanced AI , 2001, Computer.

[6]  Juan Julián Merelo Guervós,et al.  Evolving the cooperative behaviour in Unreal™ bots , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[7]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[8]  Sung-Bae Cho,et al.  Evolving Reactive NPCs for the Real-Time Simulation Game , 2005, CIG.

[9]  Sushil J. Louis,et al.  Coevolving team tactics for a real-time strategy game , 2010, IEEE Congress on Evolutionary Computation.

[10]  Stefan J. Johansson,et al.  A Multi-Agent Potential Field-based Bot for a Full RTS Game Scenario , 2009, AIIDE.

[11]  Anna I. Esparcia-Alcázar,et al.  Controlling bots in a First Person Shooter game using genetic algorithms , 2010, IEEE Congress on Evolutionary Computation.

[12]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 : Advanced Algorithms and Operators , 2000 .

[13]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[14]  C. Miles.  Co-evolving Real-Time Strategy Game Playing Influence Map Trees With Genetic Algorithms , 2022 .

[15]  Sung-Bae Cho,et al.  Optimal strategy selection of non-player character on real time strategy game using a speciated evolutionary algorithm , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[16]  Michael Buro,et al.  Call for AI Research in RTS Games , 2004 .

[17]  Juan Julián Merelo Guervós,et al.  Optimizing player behavior in a real-time strategy game using evolutionary algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[18]  Martinez Moises,et al.  Pac-mAnt: Optimization based on ant colonies applied to developing an agent for Ms. Pac-Man , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[19]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[20]  Clare Bates Congdon,et al.  Agent Smith: Towards an evolutionary rule-based agent for interactive dynamic games , 2009, 2009 IEEE Congress on Evolutionary Computation.

[21]  Penelope Sweetser.  Emergence in Games , 2007 .

[22]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[23]  Nicola Beume,et al.  Intelligent anti-grouping in real-time strategy games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[24]  David B. Fogel,et al.  Evolution-ary Computation 1: Basic Algorithms and Operators , 2000 .

[25]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[26]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[27]  David W. Aha,et al.  Automatically Generating Game Tactics through Evolutionary Learning , 2006, AI Mag..

[28]  Peter Ross,et al.  Dynamic Strategies in a Real-Time Strategy Game , 2003, GECCO.

[29]  Colm O'Riordan,et al.  Evolving Coordination for Real-Time Strategy Games , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[30]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[31]  Julian Togelius,et al.  Super mario evolution , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[32]  Juan Julián Merelo Guervós,et al.  Optimizing worst-case scenario in evolutionary solutions to the MasterMind puzzle , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[33]  Colm O'Riordan,et al.  Evolving robust strategies for an abstract real-time strategy game , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[34]  Pieter Spronck,et al.  IMPROVING OPPONENT INTELLIGENCE THROUGH OFFLINE EVOLUTIONARY LEARNING , 2003 .

[35]  Daniel Livingstone,et al.  Coevolution in Hierarchical AI for Strategy Games , 2005, CIG.

[36]  Vicente Milanés Montero,et al.  A modular parametric architecture for the TORCS racing engine , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[37]  Simon Colton,et al.  Initial Results from Co-operative Co-evolution for Automated Platformer Design , 2012, EvoApplications.

[38]  アール スピード エレック.  Game for Artificial Intelligence , 2010 .