Non stationary operator selection with island models

The purpose of adaptive operator selection is to choose dynamically the most suitable variation operator of an evolutionary algorithm at each iteration of the search process. These variation operators are applied on individuals of a population which evolves, according to an evolutionary process, in order to find an optimal solution. Of course the efficiency of an operator may change during the search and therefore its application should be precisely controlled. In this paper, we use dynamic island models as operator selection mechanisms. A sub-population is associated to each operators and individuals are allowed to migrate from one sub-population to another one. In order to evaluate the performance of this adaptive selection mechanism, we propose an abstract operator representation using fitness improvement distributions that allow us to define non stationary operators with mutual interactions. Our purpose is to show that the adaptive selection is able to identify not only good operators but also suitable sequences of operators.

[1]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[2]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[3]  A. E. Eiben,et al.  Efficient relevance estimation and value calibration of evolutionary algorithm parameters , 2007, 2007 IEEE Congress on Evolutionary Computation.

[4]  Michèle Sebag,et al.  Adaptive operator selection with dynamic multi-armed bandits , 2008, GECCO '08.

[5]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[6]  Dorian Feldman.  Contributions to the "Two-Armed Bandit" Problem , 1962 .

[7]  Darrell Whitley,et al.  The Island Model Genetic Algorithm: On Separability, Population Size and Convergence , 2015, CIT 2015.

[8]  L. Rodman.  On the Many-armed Bandit Problem , 1978 .

[9]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[10]  Frédéric Saubion,et al.  Autonomous operator management for evolutionary algorithms , 2010, J. Heuristics.

[11]  Michèle Sebag,et al.  Extreme Value Based Adaptive Operator Selection , 2008, PPSN.

[12]  Zbigniew Michalewicz,et al.  Parameter Setting in Evolutionary Algorithms , 2007, Studies in Computational Intelligence.

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Dirk Thierens,et al.  An Adaptive Pursuit Strategy for Allocating Operator Probabilities , 2005, BNAIC.

[15]  Silvestre Fialho,et al.  Adaptive operator selection for optimization , 2010 .

[16]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[17]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[18]  Michèle Sebag,et al.  Extreme compass and Dynamic Multi-Armed Bandits for Adaptive Operator Selection , 2009, 2009 IEEE Congress on Evolutionary Computation.

[19]  Zbigniew Skolicki,et al.  The influence of migration sizes and intervals on island models , 2005, GECCO '05.

[20]  Kent McClymont,et al.  Markov chain hyper-heuristic (MCHH): an online selective hyper-heuristic for multi-objective continuous problems , 2011, GECCO '11.

[21]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[22]  Frédéric Saubion,et al.  A dynamic island model for adaptive operator selection , 2012, GECCO '12.