Performance analysis of synchronous and asynchronous distributed genetic algorithms on multiprocessors

Abstract Because of their effectiveness and flexibility in finding useful solutions, Genetic Algorithms (GAs) are very popular search techniques for solving complex optimization problems in scientific and industrial fields. Parallel GAs (PGAs), and especially distributed ones have been usually presented as the way to overcome the time-consuming shortcoming of sequential GAs. In the case of applying PGAs, we can expect better performance, the reason being the exchange of knowledge during the parallel search process. The resulting distributed search is different compared to what sequential panmictic GAs do, then deserving additional studies. This article presents a performance study of three different PGAs. Moreover, we investigate the effect of synchronizing communications over modern shared-memory multiprocessors. We consider the master-slave model along with synchronous and asynchronous distributed GAs (dGAs), presenting their different designs and expected similarities when running in a number of cores ranging from one to 32 cores. The master-slave model showed a competitive numerical effort versus the other dGAs and demonstrated to be able to scale-up well over multiprocessors. We describe how the speed-up and parallel performance of the dGAs is changing as the number of cores enlarges. Results of the island model show that synchronous and asynchronous dGAs have different numerical performances on a multiprocessor, the asynchronous algorithm having a faster execution, thus more attractive for time demanding applications. Our results and statistical analyses help in developing a novel body of knowledge on PGAs running in shared memory multiprocessors (versus overwhelming literature oriented to distributed memory clusters), something useful for researchers, beginners, and final users of these techniques.

[1]  Dario Izzo,et al.  On the impact of the migration topology on the Island Model , 2010, Parallel Comput..

[2]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[3]  Enrique Alba,et al.  Improving flexibility and efficiency by adding parallelism to genetic algorithms , 2002, Stat. Comput..

[4]  Byung-Il Koh,et al.  Parallel asynchronous particle swarm optimization , 2006, International journal for numerical methods in engineering.

[5]  Enrique Alba,et al.  Influence of the Migration Policy in Parallel Distributed GAs with Structured and Panmictic Populations , 2000, Applied Intelligence.

[6]  Enrique Alba,et al.  Parallel Genetic Algorithms , 2011, Studies in Computational Intelligence.

[7]  T Watson Layne,et al.  A Distributed Genetic Algorithm with Migration for the Design of Composite Laminate Structures , 1998 .

[8]  Qingfu Zhang,et al.  Distributed evolutionary algorithms and their models: A survey of the state-of-the-art , 2015, Appl. Soft Comput..

[9]  Xin-She Yang,et al.  Bio-inspired computation: Where we stand and what's next , 2019, Swarm Evol. Comput..

[10]  Shaowen Wang,et al.  A scalable parallel genetic algorithm for the Generalized Assignment Problem , 2015, Parallel Comput..

[11]  Enrique Alba,et al.  Speed-up of synchronous and asynchronous distributed Genetic Algorithms: A first common approach on multiprocessors , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[12]  Xin Yao,et al.  Turning High-Dimensional Optimization Into Computationally Expensive Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[13]  Robert Michael Kirby,et al.  Parallel Scientific Computing in C++ and MPI - A Seamless Approach to Parallel Algorithms and their Implementation , 2003 .

[14]  Jaroslaw Sobieszczanski-Sobieski,et al.  A Parallel Particle Swarm Optimization Algorithm Accelerated by Asynchronous Evaluations , 2005 .

[15]  F. MacWilliams,et al.  The Theory of Error-Correcting Codes , 1977 .

[16]  Marc Parizeau,et al.  Analysis of a master-slave architecture for distributed evolutionary computations , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Keiki Takadama,et al.  Performance comparison of parallel asynchronous multi-objective evolutionary algorithm with different asynchrony , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[18]  Enrique Alba,et al.  A component-based study of energy consumption for sequential and parallel genetic algorithms , 2019, The Journal of Supercomputing.

[19]  Yu-Chu Tian,et al.  Theoretical Results of QoS-Guaranteed Resource Scaling for Cloud-Based MapReduce , 2018, IEEE Transactions on Cloud Computing.

[20]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[21]  Xin Yao,et al.  Negatively Correlated Search , 2015, IEEE Journal on Selected Areas in Communications.

[22]  David E. Goldberg,et al.  Efficient Parallel Genetic Algorithms: Theory and Practice , 2000 .

[23]  Enrique Alba,et al.  Parallel heterogeneous genetic algorithms for continuous optimization , 2004, Parallel Comput..

[24]  Abdel-Rahman Hedar,et al.  Advanced Parallel Genetic Algorithm with Gene Matrix for Global Optimization , 2012, AMLTA.

[25]  Kenneth A. De Jong,et al.  Using Problem Generators to Explore the Effects of Epistasis , 1997, ICGA.

[26]  Martín Pedemonte,et al.  A theoretical and empirical study of the trajectories of solutions on the grid of Systolic Genetic Search , 2018, Inf. Sci..

[27]  Anany Levitin,et al.  Introduction to the Design and Analysis of Algorithms , 2002 .

[28]  Enrique Alba,et al.  Parallel evolutionary algorithms can achieve super-linear performance , 2002, Inf. Process. Lett..

[29]  Jun Zhang,et al.  Cloudde: A Heterogeneous Differential Evolution Algorithm and Its Distributed Cloud Version , 2017, IEEE Transactions on Parallel and Distributed Systems.

[30]  Andrew Lewis,et al.  Parallel multi-objective optimization using Master-Slave model on heterogeneous resources , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[31]  Andrea Serani,et al.  Parameter selection in synchronous and asynchronous deterministic particle swarm optimization for ship hydrodynamics problems , 2016, Appl. Soft Comput..

[32]  Samuel Xavier de Souza,et al.  Parallel synchronous and asynchronous coupled simulated annealing , 2018, The Journal of Supercomputing.

[33]  Francisco Herrera,et al.  Memetic algorithms based on local search chains for large scale continuous optimisation problems: MA-SSW-Chains , 2011, Soft Comput..

[34]  Enrique Alba,et al.  Analyzing synchronous and asynchronous parallel distributed genetic algorithms , 2001, Future Gener. Comput. Syst..

[35]  Winston Khoon Guan Seah,et al.  A performance study on synchronicity and neighborhood size in particle swarm optimization , 2013, Soft Comput..

[36]  Enrique Alba,et al.  A survey of parallel distributed genetic algorithms , 1999 .

[37]  Miguel A. Vega-Rodríguez,et al.  Asynchronous Non-Generational Model to Parallelize Metaheuristics: A Bioinformatics Case Study , 2017, IEEE Transactions on Parallel and Distributed Systems.

[38]  Zong Woo Geem,et al.  A comparison study of harmony search and genetic algorithm for the max-cut problem , 2018, Swarm Evol. Comput..

[39]  Yang Yu,et al.  Parallel Pareto Optimization for Subset Selection , 2016, IJCAI.

[40]  Domingo Giménez,et al.  Optimizing Shared-memory Hyperheuristics on Top of Parameterized Metaheuristics , 2014, ICCS.

[41]  Zaher Mahjoub,et al.  On a parallel genetic-tabu search based algorithm for solving the graph colouring problem , 2009, Eur. J. Oper. Res..

[42]  Enrique Alba,et al.  Parallel Metaheuristics: A New Class of Algorithms , 2005 .

[43]  Cameron Hughes,et al.  Parallel and distributed programming using C , 2003 .

[44]  Francisco Luna,et al.  Advances in parallel heterogeneous genetic algorithms for continuous optimization , 2004 .

[45]  Ling Chen,et al.  A parallel ant colony algorithm on massively parallel processors and its convergence analysis for the travelling salesman problem , 2012, Inf. Sci..