It's Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots

Evolutionary Algorithms (EAs) are frequently used as a mechanism for the optimization of autonomous agents in games (bots), but knowing when to stop the evolution, when the bots are good enough, is not as easy as it would a priori seem. The first issue is that optimal bots are either unknown (and thus unusable as termination condition) or unreachable. In most EAs trying to find optimal bots fitness is evaluated through game playing. Many times it is found to be noisy, making its use as a termination condition also complicated. A fixed amount of evaluations or, in the case of games, a certain level of victories does not guarantee an optimal result. Thus the main objective of this paper is to test several termination conditions in order to find the one that yields optimal solutions within a restricted amount of time, and that allows researchers to compare different EAs as fairly as possible. To achieve this we will examine several ways of finishing an EA who is finding an optimal bot design process for a particular game, Planet Wars in this case, with the characteristics described above, determining the capabilities of every one of them and, eventually, selecting one for future designs.

[1]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[2]  Jürgen Branke,et al.  Creating Robust Solutions by Means of Evolutionary Algorithms , 1998, PPSN.

[3]  Debora Gil,et al.  Detecting Loss of Diversity for an Efficient Termination of EAs , 2013, 2013 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[4]  Md. Rafiqul Islam,et al.  Uncertainty and evolutionary optimization: A novel approach , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[5]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[6]  Jessica Andrea Carballido,et al.  On Stopping Criteria for Genetic Algorithms , 2004, SBIA.

[7]  N. Stander,et al.  A non‐dominance‐based online stopping criterion for multi‐objective evolutionary algorithms , 2010 .

[8]  Anna Esparcia-Alcázar,et al.  Fitness approximation for bot evolution in genetic programming , 2013, Soft Comput..

[9]  Juan Julián Merelo Guervós,et al.  Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using Evolutionary Algorithms , 2012, Journal of Computer Science and Technology.

[10]  Juan Julián Merelo Guervós,et al.  Optimizing player behavior in a real-time strategy game using evolutionary algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[11]  Robin Harper.  Evolving Robocode tanks for Evo Robocode , 2014, Genetic Programming and Evolvable Machines.

[12]  Juan Julián Merelo Guervós,et al.  Evolving Bot AI in UnrealTM , 2010, EvoApplications.

[13]  Juan Julián Merelo Guervós,et al.  Tree Depth Influence in Genetic Programming for Generation of Competitive Agents for RTS Games , 2014, EvoApplications.

[14]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[15]  Juan Julián Merelo Guervós,et al.  Genetic evolution of fuzzy finite state machines to control bots in a first-person shooter game , 2010, GECCO '10.

[16]  Carlos Cotta,et al.  Virtual player design using self-learning via competitive coevolutionary algorithms , 2014, Natural Computing.

[17]  Heike Trautmann,et al.  Online convergence detection for evolutionary multi-objective algorithms revisited , 2010, IEEE Congress on Evolutionary Computation.

[18]  T. Back,et al.  Thresholding-a selection operator for noisy ES , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[19]  Dalila Megherbi,et al.  Implementation of a parallel Genetic Algorithm on a cluster of workstations: Traveling Salesman Problem, a case study , 2001, Future Gener. Comput. Syst..

[20]  Jürgen Branke.  Searching for Robust Solutions , 2002 .

[21]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[22]  Antonio J. Fernández,et al.  Decision Tree-Based Algorithms for Implementing Bot AI in UT2004 , 2011, IWINAC.

[23]  Carlos Cotta,et al.  On balance and dynamism in procedural content generation with self-adaptive evolutionary algorithms , 2014, Natural Computing.

[24]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[25]  Juan Julián Merelo Guervós,et al.  Studying and Tackling Noisy Fitness in Evolutionary Design of Game Characters , 2014, IJCCI.

[26]  Scott B. Baden,et al.  Analysis of the numerical effects of parallelism on a parallel genetic algorithm , 1996, Proceedings of International Conference on Parallel Processing.

[27]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[28]  Clare Bates Congdon,et al.  Agent Smith: Towards an evolutionary rule-based agent for interactive dynamic games , 2009, 2009 IEEE Congress on Evolutionary Computation.

[29]  Juan Julián Merelo Guervós,et al.  Adaptive bots for real-time strategy games via map characterization , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).