The Crowding Approach to Niching in Genetic Algorithms

A wide range of niching techniques have been investigated in evolutionary and genetic algorithms. In this article, we focus on niching using crowding techniques in the context of what we call local tournament algorithms. In addition to deterministic and probabilistic crowding, the family of local tournament algorithms includes the Metropolis algorithm, simulated annealing, restricted tournament selection, and parallel recombinative simulated annealing. We describe an algorithmic and analytical framework which is applicable to a wide range of crowding algorithms. As an example of utilizing this framework, we present and analyze the probabilistic crowding niching algorithm. Like the closely related deterministic crowding approach, probabilistic crowding is fast, simple, and requires no parameters beyond those of classical genetic algorithms. In probabilistic crowding, subpopulations are maintained reliably, and we show that it is possible to analyze and predict how this maintenance takes place. We also provide novel results for deterministic crowding, show how different crowding replacement rules can be combined in portfolios, and discuss population sizing. Our analysis is backed up by experiments that further increase the understanding of probabilistic crowding.

[1]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[2]  Kalyanmoy Deb,et al.  Comparison of multi-modal optimization algorithms based on evolutionary algorithms , 2006, GECCO.

[3]  Jonathan E. Rowe,et al.  A Reduced Markov Model of GAs Without the Exact Transition Matrix , 2004, PPSN.

[4]  Gerard T. Barkema,et al.  Monte Carlo Methods in Statistical Physics , 1999 .

[5]  Pedro J. Ballester,et al.  An Effective Real-Parameter Genetic Algorithm with Parent Centric Normal Crossover for Multimodal Optimisation , 2004, GECCO.

[6]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[7]  Joseph Culberson,et al.  Genetic Invariance: A New Paradigm for Genetic Algorithm Design , 1992 .

[8]  Ole J. Mengshoel,et al.  Understanding the role of noise in stochastic local search: Analysis and experiments , 2008, Artif. Intell..

[9]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[10]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[11]  Bart Selman,et al.  Noise Strategies for Improving Local Search , 1994, AAAI.

[12]  Alain Pétrowski,et al.  A clearing procedure as a niching method for genetic algorithms , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[13]  David E. Goldberg,et al.  Probabilistic Crowding: Deterministic Crowding with Probabilistic Replacement , 1999 .

[14]  Joe Suzuki,et al.  A Markov chain analysis on simple genetic algorithms , 1995, IEEE Trans. Syst. Man Cybern..

[15]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[16]  Pedro J. Ballester,et al.  Real-Parameter Genetic Algorithms for Finding Multiple Optimal Solutions in Multi-modal Optimization , 2003, GECCO.

[17]  Pedro J. Ballester,et al.  Characterising the parameter space of a highly nonlinear inverse problem , 2006 .

[18]  Arthur C. Sanderson,et al.  Multimodal Function Optimization Using Minimal Representation Size Clustering and Its Application to Planning Multipaths , 1997, Evolutionary Computation.

[19]  David C. Wilkins,et al.  Genetic Algorithms for Belief Network Inference: The Role of Scaling and Niching , 1998, Evolutionary Programming.

[20]  Kalyanmoy Deb,et al.  Analyzing Deception in Trap Functions , 1992, FOGA.

[21]  Xin Yao,et al.  Every Niching Method has its Niche: Fitness Sharing and Implicit Sharing Compared , 1996, PPSN.

[22]  David E. Goldberg,et al.  Parallel Recombinative Simulated Annealing: A Genetic Algorithm , 1995, Parallel Comput..

[23]  Holger H. Hoos,et al.  A mixture-model for the behaviour of SLS algorithms for SAT , 2002, AAAI/IAAI.

[24]  David E. Goldberg,et al.  Finite Markov Chain Analysis of Genetic Algorithms , 1987, ICGA.

[25]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[26]  D. Goldberg,et al.  Adaptive Niching via coevolutionary Sharing , 1997 .

[27]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[28]  Kenneth A. De Jong,et al.  Analyzing GAs Using Markov Models with Semantically Ordered and Lumped States , 1996, FOGA.

[29]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[30]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[31]  David E. Goldberg,et al.  A Note on Boltzmann Tournament Selection for Genetic Algorithms and Population-Oriented Simulated Annealing , 1990, Complex Syst..

[32]  Shigenobu Kobayashi,et al.  Adaptive isolation model using data clustering for multimodal function optimization , 2005, GECCO '05.

[33]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[34]  Shigenobu Kobayashi,et al.  Sample based crowding method for multimodal optimization in continuous domain , 2005, 2005 IEEE Congress on Evolutionary Computation.

[35]  Erick Cantú-Paz,et al.  Markov chain models of parallel genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[36]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[37]  Hussein A. Abbass,et al.  Sub-structural niching in estimation of distribution algorithms , 2005, GECCO '05.

[38]  Jonathan E. Rowe,et al.  Population aggregation based on fitness , 2004, Natural Computing.

[39]  David C. Wilkins,et al.  Efficient Bayesian Network Inference: Genetic Algorithms, Stochastic Local Search, and Abstraction , 1999 .

[40]  Michael D. Vose,et al.  Modeling genetic algorithms with Markov chains , 1992, Annals of Mathematics and Artificial Intelligence.

[41]  Xiaodong Yin,et al.  A Fast Genetic Algorithm with Sharing Scheme Using Cluster Analysis Methods in Multimodal Function Optimization , 1993 .

[42]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[43]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[44]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[45]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[46]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[47]  Dirk Thierens,et al.  Elitist recombination: an integrated selection recombination GA , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[48]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[49]  E. Cantu-Paz,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1997, Evolutionary Computation.