The Uncertainty Quandary: A Study in the Context of the Evolutionary Optimization in Games and Other Uncertain Environments

In many optimization processes, the fitness or the considered measure of goodness for the candidate solutions presents uncertainty, that is, it yields different values when repeatedly measured, due to the nature of the evaluation process or the solution itself. This happens quite often in the context of computational intelligence in games, when either bots behave stochastically, or the target game possesses intrinsic random elements, but it shows up also in other problems as long as there is some random component. Thus, it is important to examine the statistical behavior of repeated measurements of performance and, more specifically, the statistical distribution that better fits them. This work analyzes four different problems related to computational intelligence in videogames, where Evolutionary Computation methods have been applied, and the evaluation of each individual is performed by playing the game, and compare them to other problem, neural network optimization, where performance is also a statistical variable. In order to find possible patterns in the statistical behavior of the variables, we track the main features of its distributions, skewness and kurtosis. Contrary to the usual assumption in this kind of problems, we prove that, in general, the values of two features imply that fitness values do not follow a normal distribution; they do present a certain common behavior that changes as evolution proceeds, getting in some cases closer to the standard distribution and in others drifting apart from it. A clear behavior in this case cannot be concluded, other than the fact that the statistical distribution that fitness variables follow is affected by selection in different directions, that parameters vary in a single generation across them, and that, in general, this kind of behavior will have to be taken into account to adequately address uncertainty in fitness in evolutionary algorithms.

[1]  Juan Julián Merelo Guervós,et al.  Evolving Multilayer Perceptrons , 2000, Neural Processing Letters.

[2]  Günter Rudolph,et al.  A partial order approach to noisy fitness functions , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[3]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[4]  Julia Handl,et al.  Implicit and Explicit Averaging Strategies for Simulation-Based Optimization of a Real-World Production Planning Problem , 2015, Informatica.

[5]  Juan Julián Merelo Guervós,et al.  Studying and Tackling Noisy Fitness in Evolutionary Design of Game Characters , 2014, IJCCI.

[6]  Juan Julián Merelo Guervós,et al.  G-Prop: Global optimization of multilayer perceptrons using GAs , 2000, Neurocomputing.

[7]  Juan Julián Merelo Guervós,et al.  Oversized Populations and Cooperative Selection: Dealing with Massive Resources in Parallel Infrastructures , 2013, LION.

[8]  Olivier Teytaud,et al.  Algorithm Portfolios for Noisy Optimization: Compare Solvers Early , 2014, LION.

[9]  Li Ming,et al.  An analysis on convergence and convergence rate estimate of elitist genetic algorithms in noisy environments , 2013 .

[10]  Benjamin W. Wah,et al.  Scheduling of Genetic Algorithms in a Noisy Environment , 1994, Evolutionary Computation.

[11]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[12]  Ignacio Rojas,et al.  Statistical analysis of the parameters of a neuro-genetic algorithm , 2002, IEEE Trans. Neural Networks.

[13]  María José del Jesús,et al.  Coevolution of lags and RBFNs for time series forecasting: L-Co-R algorithm , 2011, Soft Computing.

[14]  Juan Julián Merelo Guervós,et al.  Optimizing Strategy Parameters in a Game Bot , 2011, IWANN.

[15]  Phillip D. Stroud,et al.  Kalman-extended genetic algorithm for search in nonstationary environments with noisy fitness evaluations , 2001, IEEE Trans. Evol. Comput..

[16]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[17]  Renato Tinós,et al.  Using explicit averaging fitness for studying the behaviour of rats in a maze , 2013, ECAL.

[18]  Antonio M. Mora,et al.  My Life as a Sim: Evolving Unique and Engaging Life Stories Using Virtual Worlds , 2014, ALIFE.

[20]  Juan Julián Merelo Guervós,et al.  Optimizing player behavior in a real-time strategy game using evolutionary algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[21]  Simon M. Lucas.  Ms Pac-Man versus ghost-team competition , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[22]  Juan Julián Merelo Guervós,et al.  A Statistical Approach to Dealing with Noisy Fitness in Evolutionary Algorithms , 2014, IJCCI.

[23]  Olivier Teytaud,et al.  A mathematically derived number of resamplings for noisy optimization , 2014, GECCO.

[24]  Andrew M. Sutton,et al.  The Benefit of Recombination in Noisy Evolutionary Search , 2015, ISAAC.

[25]  Juan Julián Merelo Guervós,et al.  Co-Evolutionary Optimization of Autonomous Agents in a Real-Time Strategy Game , 2014, EvoApplications.

[26]  Yew-Soon Ong,et al.  Curse and Blessing of Uncertainty in Evolutionary Algorithm Using Approximation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[27]  Pratyusha Rakshit,et al.  Artificial Bee Colony induced multi-objective optimization in presence of noise , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[28]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[29]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[30]  Juan Julián Merelo Guervós,et al.  Comparing Heterogeneous and Homogeneous Flocking Strategies for the Ghost Team in the Game of Ms. Pac-Man , 2016, IEEE Transactions on Computational Intelligence and AI in Games.

[31]  Richard A. Groeneveld,et al.  Measuring Skewness and Kurtosis , 1984 .

[32]  Juan Julián Merelo Guervós,et al.  Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using Evolutionary Algorithms , 2012, Journal of Computer Science and Technology.

[33]  David E. Goldberg,et al.  Genetic Algorithms, Selection Schemes, and the Varying Effects of Noise , 1996, Evolutionary Computation.

[34]  Alcherio Martinoli,et al.  A distributed noise-resistant Particle Swarm Optimization algorithm for high-dimensional multi-robot learning , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[35]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[36]  Yang Yu,et al.  On the Effectiveness of Sampling for Evolutionary Optimization in Noisy Environments , 2014, PPSN.

[37]  Dirk V. Arnold,et al.  Evolution strategies in noisy environments- a survey of existing work , 2001 .

[38]  Kay Chen Tan,et al.  An Investigation on Noisy Environments in Evolutionary Multiobjective Optimization , 2007, IEEE Transactions on Evolutionary Computation.

[39]  Andrew M. Sutton,et al.  The Benefit of Sex in Noisy Evolutionary Search , 2015, ArXiv.

[40]  Leonardo Maria Reyneri,et al.  A comparison of neural networks, linear controllers, genetic algorithms and simulated annealing for real time control , 1994, ESANN.

[41]  Md. Rafiqul Islam,et al.  Uncertainty and evolutionary optimization: A novel approach , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[42]  Jan Paredis,et al.  Coevolutionary computation , 1995 .

[43]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[44]  Juan Julián Merelo Guervós,et al.  Evolving Bot AI in UnrealTM , 2010, EvoApplications.

[45]  Giovanni Squillero,et al.  MicroGP—An Evolutionary Assembly Program Generator , 2005, Genetic Programming and Evolvable Machines.

[46]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[47]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[48]  Mengjie Zhang,et al.  Population statistics for particle swarm optimization: Resampling methods in noisy optimization problems , 2014, Swarm Evol. Comput..

[49]  Zhi-Hua Zhou,et al.  Analyzing Evolutionary Optimization in Noisy Environments , 2013, Evolutionary Computation.

[50]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[51]  Vasant Honavar,et al.  Optimization of Classifiers Using Genetic Algorithms , 2001 .

[52]  Juan Julián Merelo Guervós,et al.  Optimizing Web Page Layout Using an Annealed Genetic Algorithm as Client-Side Script , 1998, PPSN.

[53]  Juan Julián Merelo Guervós,et al.  Towards automatic StarCraft strategy generation using genetic programming , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[54]  Anne Auger,et al.  Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009 , 2010, GECCO '10.

[55]  Magnus Rattray,et al.  Noisy Fitness Evaluation in Genetic Algorithms and the Dynamics of Learning , 1996, FOGA.

[56]  Juan Julián Merelo Guervós,et al.  G-Prop-III: Global Optimization of Multilayer Perceptrons using an Evolutionary Algorithm , 1999, GECCO.