Co-Evolutionary Optimization of Autonomous Agents in a Real-Time Strategy Game

This paper presents an approach based in an evolutionary algorithm, aimed to improve the behavioral parameters which guide the actions of an autonomous agent (bot) inside the real-time strategy game, Planet Wars. The work describes a co-evolutionary implementation of a previously presented method GeneBot, which yielded successful results, but focused in 4vs matches this time. Thus, there have been analyzed the effects of considering several individuals to be evolved (improved) at the same time in the algorithm, along with the use of three different fitness functions measuring the goodness of each bot in the evaluation. They are based in turns and position, and also in mathematical computations of linear regression and area regarding the number of ships belonging to the bot/individual to be evaluated. In addition, the variance of using an evolutionary algorithm with and without previous knowledge in the co-evolution phase is also studied, i.e., respectively using specific rivals to perform the evaluation, or just considering to this end individuals in the population being evolved. The aim of these co-evolutionary approaches are mainly two: first, reduce the computational time; and second find a robust fitness function to be used in the generation of evolutionary bots optimized for 4vs battles.

[1]  David W. Aha,et al.  Automatically Generating Game Tactics through Evolutionary Learning , 2006, AI Mag..

[2]  Sushil J. Louis,et al.  Using co-evolved RTS opponents to teach spatial tactics , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[3]  Sung-Bae Cho,et al.  Optimal strategy selection of non-player character on real time strategy game using a speciated evolutionary algorithm , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[4]  Juan Julián Merelo Guervós,et al.  Adaptive bots for real-time strategy games via map characterization , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[5]  Juan Julián Merelo Guervós,et al.  Optimizing Strategy Parameters in a Game Bot , 2011, IWANN.

[6]  Daniel Livingstone,et al.  Coevolution in Hierarchical AI for Strategy Games , 2005, CIG.

[7]  Juan Julián Merelo Guervós,et al.  Optimizing player behavior in a real-time strategy game using evolutionary algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[8]  Simon Colton,et al.  Initial Results from Co-operative Co-evolution for Automated Platformer Design , 2012, EvoApplications.

[9]  Sushil J. Louis,et al.  Coevolving team tactics for a real-time strategy game , 2010, IEEE Congress on Evolutionary Computation.

[10]  Jan Paredis,et al.  Coevolutionary computation , 1995 .

[11]  Carlos Cotta,et al.  An Analysis of Hall-of-Fame Strategies in Competitive Coevolutionary Algorithms for Self-Learning in RTS Games , 2013, LION.

[12]  Simon M. Lucas,et al.  Coevolution versus self-play temporal difference learning for acquiring position evaluation in small-board go , 2005, IEEE Transactions on Evolutionary Computation.

[13]  Julian Togelius,et al.  Multi-population competitive co-evolution of car racing controllers , 2007, 2007 IEEE Congress on Evolutionary Computation.

[14]  Juan Julián Merelo Guervós,et al.  Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using Evolutionary Algorithms , 2012, Journal of Computer Science and Technology.

[15]  Reinhard Männer,et al.  Parallel Problem Solving from Nature — PPSN III , 1994, Lecture Notes in Computer Science.

[16]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[17]  Jordan B. Pollack,et al.  Co-Evolution in the Successful Learning of Backgammon Strategy , 1998, Machine Learning.

[18]  John Levine,et al.  Evolution of counter-strategies: Application of co-evolution to Texas Hold'em Poker , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[19]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[20]  Zbigniew Michalewicz,et al.  Adapting to human game play , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[21]  J. Togelius,et al.  Competitive coevolution in Ms. Pac-Man , 2013, 2013 IEEE Congress on Evolutionary Computation.