Benefits of a Population: Five Mechanisms That Advantage Population-Based Algorithms

This paper identifies five distinct mechanisms by which a population-based algorithm might have an advantage over a solo-search algorithm in classical optimization. These mechanisms are illustrated through a number of toy problems. Simulations are presented comparing different search algorithms on these problems. The plausibility of these mechanisms occurring in classical optimization problems is discussed. The first mechanism we consider relies on putting together building blocks from different solutions. This is extended to include problems containing critical variables. The second mechanism is the result of focusing of the search caused by crossover. Also discussed in this context is strong focusing produced by averaging many solutions. The next mechanism to be examined is the ability of a population to act as a low-pass filter of the landscape, ignoring local distractions. The fourth mechanism is a population's ability to search different parts of the fitness landscape, thus hedging against bad luck in the initial position or the decisions it makes. The final mechanism is the opportunity of learning useful parameter values to balance exploration against exploitation.

[1]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[2]  Adam Prügel-Bennett,et al.  Modeling crossover-induced linkage in genetic algorithms , 2001, IEEE Trans. Evol. Comput..

[3]  Alden H. Wright,et al.  Emergent Behaviour, Population-based Search and Low-pass Filtering , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[4]  Emanuel Falkenauer,et al.  A hybrid grouping genetic algorithm for bin packing , 1996, J. Heuristics.

[5]  Jin-Kao Hao,et al.  Hybrid Evolutionary Algorithms for Graph Coloring , 1999, J. Comb. Optim..

[6]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[7]  Ingo Wegener,et al.  On the Analysis of Evolutionary Algorithms , 2001 .

[8]  M. Karplus,et al.  The topology of multidimensional potential energy surfaces: Theory and application to peptide structure and kinetics , 1997 .

[9]  Adam Prügel-Bennett,et al.  Symmetry breaking in population-based optimization , 2004, IEEE Transactions on Evolutionary Computation.

[10]  Nicholas J. Radcliffe,et al.  Forma Analysis and Random Respectful Recombination , 1991, ICGA.

[11]  Alain Hertz,et al.  Embedding a sequential procedure within an evolutionary algorithm for coloring problems in graphs , 1995, J. Heuristics.

[12]  Adam Pr.  Finite Population Effects for Ranking and Tournament Selection , 2000 .

[13]  Thomas Jansen,et al.  A building-block royal road where crossover is provably essential , 2007, GECCO '07.

[14]  Bart Selman,et al.  Backdoors To Typical Case Complexity , 2003, IJCAI.

[15]  Ingo Wegener,et al.  Real royal road functions--where crossover provably is essential , 2001, Discret. Appl. Math..

[16]  H. Horner.  Dynamics of learning for the binary perceptron problem , 1992 .

[17]  Adam Prügel-Bennett,et al.  Learning the Large-Scale Structure of the MAX-SAT Landscape Using Populations , 2010, IEEE Transactions on Evolutionary Computation.

[18]  Adam Prügel-Bennett,et al.  The Mixing Rate of Different Crossover Operators , 2000, FOGA.

[19]  Celia A. Glass,et al.  Genetic Algorithm for Graph Coloring: Exploration of Galinier and Hao's Algorithm , 2003, J. Comb. Optim..

[20]  John H. Holland,et al.  When will a Genetic Algorithm Outperform Hill Climbing , 1993, NIPS.

[21]  Richard A. Watson,et al.  Analysis of recombinative algorithms on a non-separable building-block problem , 2000, FOGA.

[22]  Heinz Mühlenbein,et al.  Evolution algorithms in combinatorial optimization , 1988, Parallel Comput..

[23]  Adam Prügel-Bennett,et al.  Large barrier trees for studying search , 2005, IEEE Transactions on Evolutionary Computation.

[24]  Bernd Freisleben,et al.  A Genetic Local Search Approach to the Quadratic Assignment Problem , 1997, ICGA.

[25]  Adam Prügel-Bennett,et al.  When a genetic algorithm outperforms hill-climbing , 2004, Theor. Comput. Sci..

[26]  Thomas Jansen,et al.  On the Analysis of Evolutionary Algorithms - A Proof That Crossover Really Can Help , 1999 .

[27]  Adam Prügel-Bennett,et al.  Optimal parameters for search using a barrier tree Markov model , 2007, Theor. Comput. Sci..

[28]  Carsten Witt,et al.  Population size versus runtime of a simple evolutionary algorithm , 2008, Theor. Comput. Sci..

[29]  Adam Prügel-Bennett,et al.  The Dynamics of a Genetic Algorithm on a Model Hard Optimization Problem , 2000, Complex Syst..

[30]  Martin Dietzfelbinger,et al.  The analysis of a recombinative hill-climber on H-IFF , 2003, IEEE Trans. Evol. Comput..

[31]  Bernd Freisleben,et al.  New Genetic Local Search Operators for the Traveling Salesman Problem , 1996, PPSN.

[32]  H. K. Patel,et al.  Computational complexity, learning rules and storage capacities: A Monte Carlo study for the binary perceptron , 1993 .

[33]  Adam Prügel-Bennett,et al.  Genetic algorithm dynamics in two-well potentials with basins and barrier , 1997 .

[34]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[35]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[36]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[37]  Toby Walsh,et al.  Backbones and Backdoors in Satisfiability , 2005, AAAI.

[38]  A. Rogers,et al.  A solvable model of a hard optimisation problem , 2001 .