Parallelism and evolutionary algorithms

This paper contains a modern vision of the parallelization techniques used for evolutionary algorithms (EAs). The work is motivated by two fundamental facts: 1) the different families of EAs have naturally converged in the last decade while parallel EAs (PEAs) are still lack of unified studies; and 2) there is a large number of improvements in these algorithms and in their parallelization that raise the need for a comprehensive survey. We stress the differences between the EA model and its parallel implementation throughout the paper. We discuss the advantages and drawbacks of PEAs. Also, successful applications are mentioned and open problems are identified. We propose potential solutions to these problems and classify the different ways in which recent results in theory and practice are helping to solve them. Finally, we provide a highly structured background relating to PEAs in order to make researchers aware of the benefits of decentralizing and parallelizing an EA.

[1]  Kenneth A. De Jong,et al.  An Analysis of the Effects of Neighborhood Size and Shape on Local Selection Algorithms , 1996, PPSN.

[2]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[3]  Leonardo Vanneschi,et al.  Studying the Influence of Communication Topology and Migration on Distributed Genetic Programming , 2001, EuroGP.

[4]  L. Darrell Whitley,et al.  GENITOR II: a distributed genetic algorithm , 1990, J. Exp. Theor. Artif. Intell..

[5]  Selim G. Akl,et al.  Design and analysis of parallel algorithms , 1985 .

[6]  Gilbert Syswerda,et al.  A Study of Reproduction in Generational and Steady State Genetic Algorithms , 1990, FOGA.

[7]  Douglas Comer,et al.  Internetworking with TCP/IP: volume III: client-server programming and applications (Windows sockets version) , 1997 .

[8]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[9]  Theodore C. Belding,et al.  The Distributed Genetic Algorithm Revisited , 1995, ICGA.

[10]  Fred W. Glover,et al.  A user's guide to tabu search , 1993, Ann. Oper. Res..

[11]  Shumeet Baluja,et al.  Structure and Performance of Fine-Grain Parallelism in Genetic Search , 1993, ICGA.

[12]  Benoit B. Mandelbrot,et al.  Fractals and Scaling in Finance , 1997 .

[13]  Heinz Mühlenbein,et al.  Parallel Genetic Algorithms in Combinatorial Optimization , 1992, Computer Science and Operations Research.

[14]  W. Martin,et al.  Population Structures C 6 . 3 Island ( migration ) models : evolutionary algorithms based on punctuated equilibria , 1997 .

[15]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[16]  David Mark Levine,et al.  A parallel genetic algorithm for the set partitioning problem , 1995 .

[17]  L. Darrell Whitley,et al.  Evaluating Evolutionary Algorithms , 1996, Artif. Intell..

[18]  Günter Rudolph,et al.  Significance of Locality and Selection Pressure in the Grand Deluge Evolutionary Algorithm , 1996, PPSN.

[19]  Bastien Chopard,et al.  Parallel Genetic Programming and its Application to Trading Model Induction , 1997, Parallel Comput..

[20]  M. A. Keane,et al.  The Design of Analog Circuits by Means of Genetic Programming , 2004 .

[21]  Erik D. Goodman,et al.  Investigating Parallel Genetic Algorithms on Job Shop Scheduling Problems , 1997, Evolutionary Programming.

[22]  Marco Tomassini,et al.  The Parallel Genetic Cellular Automata: Application to Global Function Optimization , 1993 .

[23]  Surya B. Yadav,et al.  The Development and Evaluation of an Improved Genetic Algorithm Based on Migration and Artificial Selection , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[24]  Antonella Carbonaro,et al.  An ANTS heuristic for the frequency assignment problem , 2000, Future Gener. Comput. Syst..

[25]  Martina Gorges-Schleuter,et al.  On the power of evolutionary optimization at the example of ATSP and large TSP Problems , 1997 .

[26]  David E. Goldberg,et al.  On the Scalability of Parallel Genetic Algorithms , 1999, Evolutionary Computation.

[27]  Masaru Tanaka,et al.  Analysis on the Island Model Parallel Genetic Algorithms for the Genetic Drifts , 1998, SEAL.

[28]  Francisco Herrera,et al.  Gradual distributed real-coded genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[29]  El-Ghazali Talbi,et al.  A Taxonomy of Hybrid Metaheuristics , 2002, J. Heuristics.

[30]  Scott B. Baden,et al.  Analysis of the numerical effects of parallelism on a parallel genetic algorithm , 1996, Proceedings of International Conference on Parallel Processing.

[31]  Günter Rudolph,et al.  A cellular genetic algorithm with self-adjusting acceptance threshold , 1995 .

[32]  Martina Gorges-Schleuter,et al.  ASPARAGOS An Asynchronous Parallel Genetic Optimization Strategy , 1989, ICGA.

[33]  David B. Fogel,et al.  A note on representations and variation operators , 1997, IEEE Trans. Evol. Comput..

[34]  Chong Fuey Sian A Java based Distributed Approach to Genetic Programming on the Internet , 1998 .

[35]  James R. Beers Book Review: Internetworking with TCP/IP Volume III: Client-Server Programming and Applications by Douglas E. Comer and David L. Stevens (Prentice Hall, 1993) , 1993, CCRV.

[36]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[37]  L. Darrell Whitley,et al.  Island Model genetic Algorithms and Linearly Separable Problems , 1997, Evolutionary Computing, AISB Workshop.

[38]  David L. Levine,et al.  Users guide to the PGAPack parallel genetic algorithm library , 1995 .

[39]  Martina Gorges-Schleuter,et al.  An Analysis of Local Selection in Evolution Strategies , 1999, GECCO.

[40]  Patrick D. Surry,et al.  The Reproductive Plan Language RPL2: Motivation, Architecture and Applications , 1994 .

[41]  David E. Goldberg,et al.  Nonstationary Function Optimization Using Genetic Algorithms with Dominance and Diploidy , 1987, ICGA.

[42]  Jason M. Daida,et al.  Biological Symbiosis as a Metaphor for Computational Hybridization , 1995, ICGA.

[43]  Bart Naudts,et al.  Epistasis as a Basic Concept in Formal Landscape Analysis , 1997, ICGA.

[44]  Markus Schwehm,et al.  Massively Parallel Genetic Algorithms , 1994, EUROSIM.

[45]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[46]  Masaharu Munetomo,et al.  An Efficient Migration Scheme for Subpopulation-Based Asynchronously Parallel Genetic Algorithms , 1993, ICGA.

[47]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[48]  Kenneth A. De Jong,et al.  An Analysis of Local Selection Algorithms in a Spatially Structured Evolutionary Algorithm , 1997, ICGA.

[49]  Andreas Zell,et al.  Median-Selection for Parallel Steady-State Evolution Strategies , 2000, PPSN.

[50]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[51]  L. Darrell Whitley,et al.  Serial and Parallel Genetic Algorithms as Function Optimizers , 1993, ICGA.

[52]  Roland Wismüller,et al.  Parallel and distributed computing , 2001, Softw. Focus.

[53]  Moshe Sipper,et al.  Evolution of Parallel Cellular Machines: The Cellular Programming Approach , 1997 .

[54]  Dirk Wiesmann,et al.  On Representation and Genetic Operators in Evolutionary Algorithms , 1998 .

[55]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[56]  Erik D. Goodman,et al.  Coarse-grain parallel genetic algorithms: categorization and new approach , 1994, Proceedings of 1994 6th IEEE Symposium on Parallel and Distributed Processing.

[57]  John R. Koza,et al.  A Parallel Implementation of Genetic Programming that Achieves Super-Linear Performance , 1998, Inf. Sci..

[58]  J. M. Troya,et al.  Stochastic reverse hill climbing and iterated local search , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[59]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[60]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[61]  Thomas Bäck,et al.  An evolutionary approach to combinatorial optimization problems , 1994, CSC '94.

[62]  Rainer Menke,et al.  A Revision of the Schema Theorem , 1998 .

[63]  Mourad Sefrioui,et al.  A Hierarchical Genetic Algorithm Using Multiple Models for Optimization , 2000, PPSN.

[64]  John R. Koza,et al.  Parallel genetic programming: a scalable implementation using the transputer network architecture , 1996 .

[65]  W Bossert,et al.  Mathematical optimization: are there abstract limits on natural selection? , 1967, The Wistar Institute symposium monograph.

[66]  Albert Y. Zomaya Parallel and Distributed Computing Handbook , 1995 .

[67]  Cesare Alippi,et al.  Genetic-algorithm programming environments , 1994, Computer.

[68]  Paul Bryant Grosso,et al.  Computer Simulations of Genetic Adaptation: Parallel Subcomponent Interaction in a Multilocus Model , 1985 .

[69]  Jens Lienig,et al.  A parallel genetic algorithm for performance-driven VLSI routing , 1997, IEEE Trans. Evol. Comput..

[70]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[71]  David E. Goldberg,et al.  Predicting Speedups of Ideal Bounding Cases of Parallel Genetic Algorithms , 1997, ICGA.

[72]  Enrique Alba,et al.  Improving flexibility and efficiency by adding parallelism to genetic algorithms , 2002, Stat. Comput..

[73]  Enrique Alba,et al.  Analyzing synchronous and asynchronous parallel distributed genetic algorithms , 2001, Future Gener. Comput. Syst..

[74]  Enrique Alba,et al.  Influence of the Migration Policy in Parallel Distributed GAs with Structured and Panmictic Populations , 2000, Applied Intelligence.

[75]  Marco Tomassini,et al.  Soft computing - integrating evolutionary, neural, and fuzzy systems , 2001 .

[76]  Pierre Kuonen,et al.  Parallel Island-Based Genetic Algorithm for Radio Network Design , 1997, J. Parallel Distributed Comput..

[77]  J. Sprave,et al.  A unified model of non-panmictic population structures in evolutionary algorithms , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[78]  Erik D. Goodman,et al.  Optimal design of flywheels using an injection island genetic algorithm , 1999, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[79]  Enrique Alba,et al.  A survey of parallel distributed genetic algorithms , 1999, Complex..

[80]  Bernard Manderick,et al.  Fine-Grained Parallel Genetic Algorithms , 1989, ICGA.

[81]  Günter Rudolph,et al.  Global Optimization by Means of Distributed Evolution Strategies , 1990, PPSN.

[82]  Francisco Herrera,et al.  Hybrid Distributed Real-Coded Genetic Algorithms , 1998, PPSN.

[83]  Patrick D. Surry,et al.  Fitness Variance of Formae and Performance Prediction , 1994, FOGA.

[84]  Vassilios Petridis,et al.  Co-operating Populations with Different Evolution Behaviours , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[85]  Al Geist,et al.  Heterogeneous parallel and distributed computing , 1999, Parallel Comput..

[86]  Vaidy S. Sunderam,et al.  PVM: A Framework for Parallel Distributed Computing , 1990, Concurr. Pract. Exp..

[87]  Douglas Comer,et al.  Internetworking with TCP/IP , 1988 .

[88]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[89]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[90]  Ron Shonkwiler,et al.  Parallel Genetic Algorithms , 1993, ICGA.

[91]  James P. Cohoon,et al.  C6.3 Island (migration) models: evolutionary algorithms based on punctuated equilibria , 1997 .

[92]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[93]  Message Passing Interface Forum MPI: A message - passing interface standard , 1994 .

[94]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[95]  L. Darrell Whitley,et al.  Cellular Genetic Algorithms , 1993, ICGA.

[96]  E. Tarantino,et al.  Testing parallel evolution strategies on the quadratic assignment problem , 1993, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[97]  Chrisila C. Pettey,et al.  A Theoretical Investigation of a Parallel Genetic Algorithm , 1989, ICGA.

[98]  C. A. Coello Coello,et al.  A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques , 1999, Knowledge and Information Systems.

[99]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[100]  Enrique Alba,et al.  Gaining New Fields of Application for OOP : the Parallel Evolutionary Algorithm Case , 2001 .

[101]  Enrique Alba,et al.  Parallel evolutionary algorithms can achieve super-linear performance , 2002, Inf. Process. Lett..

[102]  El-Ghazali Talbi,et al.  A multiobjective genetic algorithm for radio network optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[103]  Amjad Umar,et al.  Object-Oriented Client/Server Internet Environments , 1997 .

[104]  Joachim Stender,et al.  Parallel Genetic Algorithms: Theory and Applications , 1993 .

[105]  Enrique Alba,et al.  Cellular Evolutionary Algorithms: Evaluating the Influence of Ratio , 2000, PPSN.

[106]  William F. Punch HOW EFFECTIVE ARE MULTIPLE POPULATIONS IN GENETIC PROGRAMMING , 1998 .

[107]  Dana S. Richards,et al.  Punctuated Equilibria: A Parallel Genetic Algorithm , 1987, ICGA.

[108]  David B. Fogel,et al.  Evolutionary algorithms in theory and practice , 1997, Complex.

[109]  Marco Tomassini,et al.  A Statistical Study of a Class of Cellular Evolutionary Algorithms , 1999, Evolutionary Computation.

[110]  Xin Yao,et al.  PEPNet: Parallel Evolutionary Programming for Constructing Artificial Neural Networks , 1997, Evolutionary Programming.

[111]  Xin Yao,et al.  Parallel evolutionary programming , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[112]  Martin Schütz,et al.  Application of Parallel Mixed-Integer Evolution Strategies with Mutation Rate Pooling , 1996, Evolutionary Programming.

[113]  M. Gorges-Schleuter Asparagos96 and the traveling salesman problem , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[114]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[115]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[116]  Marco Tomassini,et al.  Experimental Study of Multipopulation Parallel Genetic Programming , 2000, EuroGP.

[117]  Giandomenico Spezzano,et al.  CAGE: A Tool for Parallel Genetic Programming Applications , 2001, EuroGP.

[118]  Andrew S. Grimshaw,et al.  The Legion vision of a worldwide virtual computer , 1997, Commun. ACM.

[119]  Erick Cantú-Paz,et al.  A Survey of Parallel Genetic Algorithms , 2000 .

[120]  D. Goldberg,et al.  Predicting Speedups of Idealized Bounding Cases of Parallel Genetic Algorithms , 1997 .

[121]  Marco Tomassini,et al.  PARALLEL AND DISTRIBUTED EVOLUTIONARY COMPUTATION FOR FINANCIAL APPLICATIONS , 2000, Parallel Algorithms Appl..

[122]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[123]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[124]  Yuval Davidor,et al.  A Naturally Occurring Niche and Species Phenomenon: The Model and First Results , 1991, ICGA.

[125]  Akihiko Konagaya,et al.  A Fine-Grained Parallel Genetic Algorithm for Distributed Parallel Systems , 1993, ICGA.

[126]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[127]  Siegfried Benkner VFC: The Vienna Fortran Compiler , 1999, Sci. Program..

[128]  Reiko Tanese,et al.  Parallel Genetic Algorithms for a Hypercube , 1987, ICGA.