Asynchronous Master-Slave Parallelization of Differential Evolution for Multi-Objective Optimization

In this paper, we present AMS-DEMO, an asynchronous master-slave implementation of DEMO, an evolutionary algorithm for multi-objective optimization. AMS-DEMO was designed for solving time-intensive problems efficiently on both homogeneous and heterogeneous parallel computer architectures. The algorithm is used as a test case for the asynchronous master-slave parallelization of multi-objective optimization that has not yet been thoroughly investigated. Selection lag is identified as the key property of the parallelization method, which explains how its behavior depends on the type of computer architecture and the number of processors. It is arrived at analytically and from the empirical results. AMS-DEMO is tested on a benchmark problem and a time-intensive industrial optimization problem, on homogeneous and heterogeneous parallel setups, providing performance results for the algorithm and an insight into the parallelization method. A comparison is also performed between AMS-DEMO and generational master-slave DEMO to demonstrate how the asynchronous parallelization method enhances the algorithm and what benefits it brings compared to the synchronous method.

[1]  Kazuhiro Nakahashi,et al.  Aerodynamic Shape Optimization of Supersonic Wings by Adaptive Range Multiobjective Genetic Algorithms , 2001, EMO.

[2]  Bogdan Filipic,et al.  DEMO: Differential Evolution for Multiobjective Optimization , 2005, EMO.

[3]  Anne Auger,et al.  Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point , 2009, FOGA '09.

[4]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[5]  Andrew Lewis,et al.  Asynchronous multiple objective particle swarm optimisation in unreliable distributed environments , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[6]  Bu-Sung Lee,et al.  Efficient Hierarchical Parallel Genetic Algorithms using Grid computing , 2007, Future Gener. Comput. Syst..

[7]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[8]  Enrique Alba,et al.  Parallel evolutionary algorithms can achieve super-linear performance , 2002, Inf. Process. Lett..

[9]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[10]  Janez Brest,et al.  Differential evolution for multiobjective optimization with self adaptation , 2007, 2007 IEEE Congress on Evolutionary Computation.

[11]  Enrique Alba,et al.  Improving flexibility and efficiency by adding parallelism to genetic algorithms , 2002, Stat. Comput..

[12]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[13]  Bogdan Filipic,et al.  Model-Based Tuning of Process Parameters for Steady-State Steel Casting , 2005, Informatica.

[14]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[15]  Enrique Alba,et al.  Parallel Evolutionary Computations , 2006, Studies in Computational Intelligence.

[16]  Günter Rudolph,et al.  Parallel Approaches for Multiobjective Optimization , 2008, Multiobjective Optimization.

[17]  El-Ghazali Talbi,et al.  Hierarchical parallel approach for GSM mobile network design , 2006, J. Parallel Distributed Comput..

[18]  Selim G. Akl.  Parallel computation: models and methods , 1997 .

[19]  Bogdan Filipič,et al.  Parallel Evolutionary Computation Framework for Single- and Multiobjective Optimization , 2009 .

[20]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[21]  Franck Cappello,et al.  Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed , 2006, Int. J. High Perform. Comput. Appl..

[22]  Gary B. Lamont,et al.  Considerations in engineering parallel multiobjective evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[23]  Peter Eberhard,et al.  Parallel Evolutionary Optimization of Multibody Systems with Application to Railway Dynamics , 2003 .

[24]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[25]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[26]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[27]  Trevor N. Mudge,et al.  A Parallel Genetic Algorithm for Multiobjective Microprocessor Design , 1995, ICGA.

[28]  D. Corne,et al.  On Metrics for Comparing Non Dominated Sets , 2001 .

[29]  Byung-Il Koh,et al.  Parallel asynchronous particle swarm optimization , 2006, International journal for numerical methods in engineering.

[30]  Francisco Luna,et al.  Parallel Multiobjective Optimization , 2005 .

[31]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[32]  Antonio J. Nebro,et al.  A Study of the Parallelization of the Multi-Objective Metaheuristic MOEA/D , 2010, LION.

[33]  R. Lyndon While,et al.  A Scalable Multi-objective Test Problem Toolkit , 2005, EMO.

[34]  Erick Cantú-Paz,et al.  A Survey of Parallel Genetic Algorithms , 2000 .

[35]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[36]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[37]  Andrew Lewis,et al.  Parallel multi-objective optimization using Master-Slave model on heterogeneous resources , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[38]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[39]  Luiz Eduardo Soares de Oliveira,et al.  A Methodology for Feature Selection Using Multiobjective Genetic Algorithms for Handwritten Digit String Recognition , 2003, Int. J. Pattern Recognit. Artif. Intell..

[40]  Luiz Eduardo Soares de Oliveira,et al.  Intelligent zoning design using multi-objective evolutionary algorithms , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[41]  Steven A. Orszag,et al.  CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS , 1978 .

[42]  Cai Dai,et al.  A New Multiobjective Evolutionary Algorithm Based on Decomposition of the Objective Space for Multiobjective Optimization , 2014, J. Appl. Math..

[43]  Anthony Skjellum,et al.  A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[44]  Enrique Alba,et al.  Parallel Evolutionary Multiobjective Optimization , 2006, Parallel Evolutionary Computations.

[45]  Tea Tusar,et al.  Preliminary Numerical Experiments in Multiobjective Optimization of a Metallurgical Production Process , 2007, Informatica.

[46]  Tea Tusar,et al.  Differential Evolution versus Genetic Algorithms in Multiobjective Optimization , 2007, EMO.

[47]  Andrew Lewis,et al.  Asynchronous Multi-Objective Optimisation in Unreliable Distributed Environments , 2009 .

[48]  Rainer Storn,et al.  Differential Evolution-A simple evolution strategy for fast optimization , 1997 .

[49]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[50]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[51]  A. Vicini,et al.  Sub-population policies for a parallel multiobjective genetic algorithm with applications to wing design , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).