Simulated Evolutionary Optimization and Local Search: Introduction and Application to Tree Search

Tree search and its more complicated variant, tree search and simultaneous multiple DNA sequence alignment, are difficult NP-complete optimization problems, which require the application of advanced computational techniques, if large data sets are to be solved within reasonable computation times. Traditionally tree search has been attacked with a search strategy that is best described as multistart hill-climbing; local search by branch swapping has been performed on several different starting trees. Recently a different tree search strategy was tested in the Parsigal parsimony program, which used a combination of evolutionary optimization and local search. Evolutionary optimization algorithms use principles adopted from biological evolution to solve technical optimization tasks. Evolutionary optimization is a stochastic global search method, which means that the method is able to escape local optima, and is in principle able to produce any solution in the search space (although this may take a long time). Local search techniques, such as branch swapping, employ a completely different search strategy; they exploit local information maximally in order to achieve quick improvement in the value of the objective function. However, local search algorithms lack the ability to escape from local optima, which is a fundamental requirement for any search algorithm that aims to be able to discover the global optimum of a multimodal optimization problem. Hence it seems that an optimization strategy combining the good properties of both evolutionary algorithms and local search would be ideal. In this study, aspects of global optimization and local search are discussed, and the method of simulated evolutionary optimization is reviewed in detail. The application of simulated evolutionary optimization to tree search in Parsigal is then reviewed briefly.

[1]  John R. Koza Proceedings of the 1st annual conference on genetic programming , 1996 .

[2]  Kevin C. Nixon Regular ArticleThe Parsimony Ratchet, a New Method for Rapid Parsimony Analysis☆ , 1999 .

[3]  Stephanie Forrest,et al.  Proceedings of the 5th International Conference on Genetic Algorithms , 1993 .

[4]  David S. Gladstein,et al.  Efficient Incremental Character Optimization , 1997, Cladistics : the international journal of the Willi Hennig Society.

[5]  Reinhard Männer,et al.  Parallel Problem Solving from Nature , 1991 .

[6]  Terry Jones,et al.  Crossover, Macromutationand, and Population-Based Search , 1995, ICGA.

[7]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[8]  Thomas Bäck,et al.  Evolutionary computation: comments on the history and current state , 1997, IEEE Trans. Evol. Comput..

[9]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[10]  Atte Moilanen,et al.  Searching for Most Parsimonious Trees with Simulated Evolutionary Optimization , 1999 .

[11]  Günter Rudolph,et al.  Contemporary Evolution Strategies , 1995, ECAL.

[12]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[13]  Phil Husbands,et al.  A Comparison of Optimization Techniques for Integrated Manufacturing Planning and Scheduling , 1996, PPSN.

[14]  Jin-Kao Hao,et al.  A New Genetic Local Search Algorithm for Graph Coloring , 1998, PPSN.

[15]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[16]  Zbigniew Michalewicz,et al.  An Experimental Comparison of Binary and Floating Point Representations in Genetic Algorithms , 1991, ICGA.

[17]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[19]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[20]  Dirk Van Gucht,et al.  The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem , 1989 .

[21]  Thomas Bäck,et al.  Parallel Problem Solving from Nature — PPSN V , 1998, Lecture Notes in Computer Science.

[22]  William E. Hart,et al.  A Comparison of Global and Local Search Methods in Drug Docking , 1997, ICGA.

[23]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[24]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[25]  Larry J. Eshelman,et al.  Crossover's Niche , 1993, ICGA.

[26]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[27]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[28]  Hugh M. Cartwright,et al.  Looking Around: Using Clues from the Data Space to Guide Genetic Algorithm Searches , 1991, ICGA.

[29]  F. Ayala Molecular systematics , 2004, Journal of Molecular Evolution.

[30]  Jarmo T. Alander,et al.  An Indexed Bibliography of Genetic Algorithms , 1995 .

[31]  Juan Julián Merelo Guervós,et al.  Solving Master Mind Using GAs and Simulated Annealing: A Case of Dynamic Constraint Optimization , 1996, PPSN.

[32]  L. Cavalli-Sforza,et al.  PHYLOGENETIC ANALYSIS: MODELS AND ESTIMATION PROCEDURES , 1967, Evolution; international journal of organic evolution.

[33]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[34]  Una-May O'Reilly,et al.  Program Search with a Hierarchical Variable Lenght Representation: Genetic Programming, Simulated Annealing and Hill Climbing , 1994, PPSN.

[35]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[36]  Hans-Paul Schwefel,et al.  Parallel Problem Solving from Nature — PPSN IV , 1996, Lecture Notes in Computer Science.

[37]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[38]  William McGinnis,et al.  Drosophila Homeobox Genes , 1993 .

[39]  D. Maddison The discovery and importance of multiple islands of most , 1991 .

[40]  A. Phillips,et al.  Multiple sequence alignment in phylogenetic analysis. , 2000, Molecular phylogenetics and evolution.

[41]  Per S. Laursen Problem-Independent Parallel Simulated Annealing using Selection and Migration , 1994, PPSN.

[42]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[43]  Gregory J. E. Rawlins,et al.  Reverse HillclimbingGenetic Algorithms and the Busy Beaver Problem , 1993, ICGA.

[44]  James P. Cohoon,et al.  Population-Oriented Simulated Annealing: A Genetic/Thermodynamic Hybrid Approach to Optimization , 1995, International Conference on Genetic Algorithms.

[45]  Hao Chen,et al.  Parallel Simulated Annealing and Genetic Algorithms: a Space of Hybrid Methods , 1994, PPSN.

[46]  Richard K. Belew,et al.  When Both Individuals and Populations Search: Adding Simple Learning to the Genetic Algorithm , 1989, ICGA.

[47]  Hitoshi Iba,et al.  Genetic Programming with Local Hill-Climbing , 1994, PPSN.

[48]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[49]  Charles E. Taylor Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. Complex Adaptive Systems.John H. Holland , 1994 .

[50]  Nachum Dershowitz The evolution of programs , 1983, Progress in computer science.

[51]  P. Goloboff METHODS FOR FASTER PARSIMONY ANALYSIS , 1996 .

[52]  Hisao Ishibuchi,et al.  Effectiveness of Genetic Local Search Algorithms , 1997, ICGA.

[53]  Larry J. Eshelman,et al.  Preventing Premature Convergence in Genetic Algorithms by Preventing Incest , 1991, ICGA.

[54]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[55]  L. Darrell Whitley,et al.  Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect , 1993, Evolutionary Computation.

[56]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[57]  Heinz Mühlenbein,et al.  The Science of Breeding and Its Application to the Breeder Genetic Algorithm (BGA) , 1993, Evolutionary Computation.

[58]  Cheng-Yan Kao,et al.  Applying the genetic approach to simulated annealing in solving some NP-hard problems , 1993, IEEE Trans. Syst. Man Cybern..

[59]  K. Nixon,et al.  The Parsimony Ratchet, a New Method for Rapid Parsimony Analysis , 1999, Cladistics : the international journal of the Willi Hennig Society.

[60]  Wirt Atmar,et al.  Notes on the simulation of evolution , 1994, IEEE Trans. Neural Networks.

[61]  John A. Miller,et al.  An evaluation of local improvement operators for genetic algorithms , 1993, IEEE Trans. Syst. Man Cybern..

[62]  Kalyanmoy Deb,et al.  Long Path Problems , 1994, PPSN.

[63]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[64]  Thomas Bäck,et al.  Optimal Mutation Rates in Genetic Search , 1993, ICGA.

[65]  Jean-Michel Renders,et al.  Hybrid methods using genetic algorithms for global optimization , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[66]  Larry J. Eshelman,et al.  Proceedings of the 6th International Conference on Genetic Algorithms , 1995 .

[67]  Takeshi Yamada,et al.  Scheduling by Genetic Local Search with Multi-Step Crossover , 1996, PPSN.

[68]  Martina Gorges-Schleuter,et al.  Comparison of Local Mating Strategies in Massively Parallel Genetic Algorithms , 1992, PPSN.

[69]  Vasant Honavar,et al.  Proceedings of the Genetic and Evolutionary Computation Conference , 2021, GECCO.

[70]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[71]  Ehl Emile Aarts,et al.  Simulated annealing and Boltzmann machines , 2003 .

[72]  Philippe Preux,et al.  Climbing Up NP-Hard Hills , 1996, PPSN.

[73]  Samir W. Mahfoud A Comparison of Parallel and Sequential Niching Methods , 1995, ICGA.

[74]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[75]  P. Goloboff Analyzing Large Data Sets in Reasonable Times: Solutions for Composite Optima , 1999, Cladistics : the international journal of the Willi Hennig Society.

[76]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[77]  L. Booker Foundations of genetic algorithms. 2: L. Darrell Whitley (Ed.), Morgan Kaufmann, San Mateo, CA, 1993, ISBN 1-55860-263-1, 322 pp., US$45.95 , 1994 .

[78]  R. Graham,et al.  Unlikelihood that minimal phylogenies for a realistic biological study can be constructed in reasonable computational time , 1982 .

[79]  Alice E. Smith,et al.  Local Search Genetic Algorithm for Optimization of Highly Reliable Communications Networks , 1997, ICGA.

[80]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[81]  L. Darrell Whitley,et al.  Lamarckian Evolution, The Baldwin Effect and Function Optimization , 1994, PPSN.

[82]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[83]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[84]  Károly F. Pál,et al.  Selection Schemes with Spatial Isolation for Genetic Optimization , 1994, PPSN.

[85]  D. Fogel Evolutionary algorithms in theory and practice , 1997, Complex..

[86]  D Cvijovicacute,et al.  Taboo search: an approach to the multiple minima problem. , 1995, Science.

[87]  Thomas Bäck,et al.  Parallel Optimization of Evolutionary Algorithms , 1994, PPSN.