Impacts of sampling strategies in tournament selection for genetic programming

Tournament selection is one of the most commonly used parent selection schemes in genetic programming (GP). While it has a number of advantages over other selection schemes, it still has some issues that need to be thoroughly investigated. Two of the issues are associated with the sampling process from the population into the tournament. The first one is the so-called “multi-sampled” issue, where some individuals in the population are picked up (sampled) many times to form a tournament. The second one is the “not-sampled” issue, meaning that some individuals are never picked up when forming tournaments. In order to develop a more effective selection scheme for GP, it is necessary to understand the actual impacts of these issues in standard tournament selection. This paper investigates the behaviour of different sampling replacement strategies through mathematical modelling, simulations and empirical experiments. The results show that different sampling replacement strategies have little impact on selection pressure and cannot effectively tune the selection pressure in dynamic evolution. In order to conduct effective parent selection in GP, research focuses should be on developing automatic and dynamic selection pressure tuning methods instead of alternative sampling replacement strategies. Although GP is used in the empirical experiments, the findings revealed in this paper are expected to be applicable to other evolutionary algorithms.

[1]  John R. Koza,et al.  Routine human-competitive machine intelligence by means of genetic programming , 2004, SPIE Optics + Photonics.

[2]  Osman Kaan Erol,et al.  A Genetic Programming Classifier Design Approach for Cell Images , 2007, ECSQARU.

[3]  Francisco Herrera,et al.  A Survey on the Application of Genetic Programming to Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  John R. Koza,et al.  Genetic Programming III - Darwinian Invention and Problem Solving , 1999, Evolutionary Computation.

[5]  Hod Lipson,et al.  Learning noise , 2007, GECCO '07.

[6]  M. Bulmer The Mathematical Theory of Quantitative Genetics , 1981 .

[7]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[8]  J. S. Hunter,et al.  Statistics for Experimenters: Design, Innovation, and Discovery , 2006 .

[9]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[10]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[11]  Sung-Bae Cho,et al.  Lymphoma Cancer Classification Using Genetic Programming with SNR Features , 2004, EuroGP.

[12]  John R. Koza,et al.  Genetic Programming III: Darwinian Invention & Problem Solving , 1999 .

[13]  John R. Koza,et al.  Genetic Programming IV: Routine Human-Competitive Machine Intelligence , 2003 .

[14]  Steven M. Gustafson An analysis of diversity in genetic programming , 2004 .

[15]  Kenneth A. De Jong,et al.  Understanding EA Dynamics via Population Fitness Distributions , 2003, GECCO.

[16]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[17]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[18]  Vic Ciesielski,et al.  Prevention of early convergence in genetic programming by replacement of similar programs , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[19]  David E. Goldberg,et al.  Influence of selection and replacement strategies on linkage learning in BOA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[20]  David J. Montana,et al.  Automated hardware design using genetic programming, VHDL, and FPGAs , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[21]  Lothar Thiele,et al.  A Mathematical Analysis of Tournament Selection , 1995, ICGA.

[22]  Wei Zhang,et al.  Genetic programming-based chaotic time series modeling , 2004, Journal of Zhejiang University. Science.

[23]  Riccardo Poli,et al.  Backward-chaining evolutionary algorithms , 2006, Artif. Intell..

[24]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[25]  Wo-Chiang Lee,et al.  Genetic Programming Decision Tree for Bankruptcy Prediction , 2006, JCIS.

[26]  Mengjie Zhang,et al.  GP for Object Classification: Brood Size in Brood Recombination Crossover , 2006, Australian Conference on Artificial Intelligence.

[27]  Arthur K. Kordon,et al.  Variable Selection in Industrial Datasets Using Pareto Genetic Programming , 2006 .

[28]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[29]  Kenneth de Jong Parameter Setting in EAs: a 30 Year Perspective , 2007 .

[30]  Thomas Bäck,et al.  Selective Pressure in Evolutionary Algorithms: A Characterization of Selection Mechanisms , 1994, International Conference on Evolutionary Computation.

[31]  Mengjie Zhang,et al.  Another investigation on tournament selection: modelling and visualisation , 2007, GECCO '07.

[32]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[33]  John J. Grefenstette,et al.  How Genetic Algorithms Work: A Critical Look at Implicit Parallelism , 1989, ICGA.

[34]  Arthur K. Kordon,et al.  Using Genetic Programming in Industrial Statistical Model Building , 2005 .

[35]  David E. Goldberg,et al.  Genetic Algorithms, Selection Schemes, and the Varying Effects of Noise , 1996, Evolutionary Computation.

[36]  E. Tsang,et al.  Reducing Failures In Investment Recommendations Using Genetic Programming , 2000 .

[37]  Mengjie Zhang,et al.  Genetic Programming for detecting rhythmic stress in spoken English , 2008, Int. J. Knowl. Based Intell. Eng. Syst..

[38]  Róbert Ványi Practical Evaluation of Efficient Fitness Functions for Binary Images , 2005, EvoWorkshops.

[39]  Irene A. Stegun,et al.  Handbook of Mathematical Functions. , 1966 .

[40]  Leonardo Bruno de Sá,et al.  Evolutionary synthesis of low-sensitivity equalizers using adjacency matrix representation , 2008, GECCO '08.

[41]  L. Darrell Whitley,et al.  Unbiased tournament selection , 2005, GECCO '05.

[42]  Wolfgang Banzhaf,et al.  A comparison of linear genetic programming and neural networks in medical data mining , 2001, IEEE Trans. Evol. Comput..

[43]  Zbigniew Michalewicz,et al.  Parameter Setting in Evolutionary Algorithms , 2007, Studies in Computational Intelligence.

[44]  Victor Ciesielski,et al.  A Domain-Independent Window Approach to Multiclass Object Detection Using Genetic Programming , 2003, EURASIP J. Adv. Signal Process..

[45]  Tatsuya Motoki,et al.  Calculating the Expected Loss of Diversity of Selection Schemes , 2002, Evolutionary Computation.

[46]  Luca Lombardi,et al.  Image classification: an evolutionary approach , 2002, Pattern Recognit. Lett..

[47]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[48]  Stephan M. Winkler,et al.  GA-Selection Revisited from an ES-Driven Point of View , 2005, IWINAC.