Studying the effect of population size in distributed evolutionary algorithms on heterogeneous clusters

Graphical abstractDisplay Omitted HighlightsWe present a study on the adaptation of the island sizes of a distributed Evolutionary Algorithm to the computational power of the nodes of an heterogeneous cluster.Three problems with different characteristics and computational demands (MMDP, OneMax and Rosenbrock Shifted Function), have been used to give an insight of the effects of this proposal and tested in two different hardware systems (homogeneous and heterogeneous cluster).As a first step, we have adapted (offline) population sizes proportionally to the attained generations to obtain the optimum with the same population size in all islands.We also have adapted the population size during runtime (online) comparing with the current number of generations of the neighbours of the topology.Results show that adapting (online and offline) the population sizes to the computational power of the nodes of a heterogeneous cluster can decrease time to obtain the optimum solution. Distributed Evolutionary Algorithms are traditionally executed on homogeneous dedicated clusters, despite most scientists have access mainly to networks of heterogeneous nodes (e.g., desktop PCs in a lab). Fitting this kind of algorithms to these environments, so that they can take advantage of their heterogeneity to save running time, is still an open problem. The different computational power of the nodes affects the performance of the algorithm, and tuning or fitting it to each node properly could reduce execution time.Since the distributed Evolutionary Algorithms include a whole range of parameters that influence the performance, this paper proposes a study on the population size. This parameter is one of the most important, since it has a direct relationship with the number of iterations needed to find the solution, as it affects the exploration factor of the algorithm. The aim of this paper consists in validating the following hypothesis: fitting the sub-population size to the computational power of the heterogeneous cluster node can lead to an improvement in running time with respect to the use of the same population size in every node.Two parameter size schemes have been tested, an offline and an online parameter setting, and three problems with different characteristics and computational demands have been used.Results show that setting the population size according to the computational power of each node in the heterogeneous cluster improves the time required to obtain the optimal solution. Meanwhile, the same set of different size values could not improve the running time to reach the optimum in a homogeneous cluster with respect to the same size in all nodes, indicating that the improvement is due to the interaction of the different hardware resources with the algorithm. In addition, a study on the influence of the different population sizes on each stage of the algorithm is presented. This opens a new research line on the fitting (offline or online) of parameters of the distributed Evolutionary Algorithms to the computational power of the devices.

[1]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[2]  David E. Goldberg,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1999, Evolutionary Computation.

[3]  Cláudio F. Lima,et al.  Adaptive Population Sizing Schemes in Genetic Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[4]  Enrique Alba,et al.  Designing heterogeneous distributed GAs by efficiently self-adapting the migration period , 2011, Applied Intelligence.

[5]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[6]  Heinz Mühlenbein,et al.  Adaptation of population sizes by competing subpopulations , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[7]  David E. Goldberg,et al.  Are Multiple Runs of Genetic Algorithms Better than One? , 2003, GECCO.

[8]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[9]  Eric Goodman,et al.  Investigations in meta-GAs: panaceas or pipe dreams? , 2005, GECCO '05.

[10]  Erick Cantú-Paz,et al.  Topologies, Migration Rates, and Multi-Population Parallel Genetic Algorithms , 1999, GECCO.

[11]  Kalyan Veeramachaneni,et al.  Cloud Driven Design of a Distributed Genetic Programming Platform , 2013, EvoApplications.

[12]  Alex S. Fukunaga,et al.  Distributed island-model genetic algorithms using heterogeneous parameter settings , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[13]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[14]  Ville Tirronen,et al.  Distributed differential evolution with explorative–exploitative population families , 2009, Genetic Programming and Evolvable Machines.

[15]  Xavier Llorà,et al.  Scaling Genetic Algorithms Using MapReduce , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[16]  Ian T. Foster.  Globus Toolkit Version 4: Software for Service-Oriented Systems , 2005, NPC.

[17]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[18]  Alex S. Fukunaga,et al.  Evaluation of a randomized parameter setting strategy for island-model evolutionary algorithms , 2013, 2013 IEEE Congress on Evolutionary Computation.

[19]  A. E. Eiben,et al.  Parameter tuning for configuring and analyzing evolutionary algorithms , 2011, Swarm Evol. Comput..

[20]  Morikazu Nakamura,et al.  Parallel genetic algorithms on line topology of heterogeneous computing resources , 2005, GECCO '05.

[21]  Francisco Herrera,et al.  Evolutionary parallel and gradually distributed lateral tuning of fuzzy rule-based systems , 2009, Evol. Intell..

[22]  Satoshi Matsuoka,et al.  Linpack evaluation on a supercomputer with heterogeneous accelerators , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[23]  Larry J. Eshelman,et al.  On Crossover as an Evolutionarily Viable Strategy , 1991, ICGA.

[24]  Enrique Alba,et al.  Evaluation of parallel metaheuristics , 2006 .

[25]  Juan Julián Merelo Guervós,et al.  Randomized Parameter Settings for Heterogeneous Workers in a Pool-Based Evolutionary Algorithm , 2014, PPSN.

[26]  Jack J. Dongarra,et al.  The LINPACK Benchmark: past, present and future , 2003, Concurr. Comput. Pract. Exp..

[27]  A. E. Eiben.  Evolutionary Computing and Autonomic Computing: Shared Problems, Shared Solutions? , 2005, Self-star Properties in Complex Information Systems.

[28]  Enrique Alba,et al.  Heterogeneous Computing and Parallel Genetic Algorithms , 2002, J. Parallel Distributed Comput..

[29]  Enrique Alba,et al.  Dealing with hardware heterogeneity: a new parallel search model , 2013, Natural Computing.

[30]  Gary B. Lamont,et al.  Load balancing for heterogeneous clusters of PCs , 2002, Future Gener. Comput. Syst..

[31]  Paul Avery,et al.  A Science Driven Production Cyberinfrastructure—the Open Science Grid , 2011, Journal of Grid Computing.

[32]  Juan Julián Merelo Guervós,et al.  Diversity Through Multiculturality: Assessing Migrant Choice Policies in an Island Model , 2011, IEEE Transactions on Evolutionary Computation.

[33]  Prabhas Chongstitvatana,et al.  Parallel genetic algorithm with parameter adaptation , 2002, Inf. Process. Lett..

[34]  Andreas Zell,et al.  EvA: a tool for optimization with evolutionary algorithms , 1997, EUROMICRO 97. Proceedings of the 23rd EUROMICRO Conference: New Frontiers of Information Technology (Cat. No.97TB100167).

[35]  Juan Julián Merelo Guervós,et al.  Service oriented evolutionary algorithms , 2013, Soft Comput..

[36]  José Luis Bosque,et al.  Parallel implementation of evolutionary strategies on heterogeneous clusters with load balancing , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[37]  Enrique Alba,et al.  Distributed Evolutionary Algorithms in Heterogeneous Environments , 2013, 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing.

[38]  Francisco Almeida,et al.  Dynamic load balancing on heterogeneous multi-GPU systems , 2013, Comput. Electr. Eng..

[39]  Juan Julián Merelo Guervós,et al.  Asynchronous distributed genetic algorithms with Javascript and JSON , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[40]  José A. Martínez,et al.  Adaptive load balancing of iterative computation on heterogeneous nondedicated systems , 2011, The Journal of Supercomputing.

[41]  Christian Gagné,et al.  Improving genetic algorithms performance via deterministic population shrinkage , 2009, GECCO.

[42]  Enrique Alba,et al.  HydroCM: A Hybrid Parallel Search Model for Heterogeneous Platforms , 2013, Hybrid Metaheuristics.

[43]  Holger H. Hoos,et al.  Automatically Configuring Algorithms for Scaling Performance , 2012, LION.

[44]  Agostinho C. Rosa,et al.  Self-regulated Population Size in Evolutionary Algorithms , 2006, PPSN.