Improving Classical and Decentralized Differential Evolution With New Mutation Operator and Population Topologies

Differential evolution (DE) algorithms compose an efficient type of evolutionary algorithm (EA) for the global optimization domain. Although it is well known that the population structure has a major influence on the behavior of EAs, there are few works studying its effect in DE algorithms. In this paper, we propose and analyze several DE variants using different panmictic and decentralized population schemes. As it happens for other EAs, we demonstrate that the population scheme has a marked influence on the behavior of DE algorithms too. Additionally, a new operator for generating the mutant vector is proposed and compared versus a classical one on all the proposed population models. After that, a new heterogeneous decentralized DE algorithm combining the two studied operators in the best performing studied population structure has been designed and evaluated. In total, 13 new DE algorithms are presented and evaluated in this paper. Summarizing our results, all the studied algorithms are highly competitive compared to the state-of-the-art DE algorithms taken from the literature for most considered problems, and the best ones implement a decentralized population. With respect to the population structure, the proposed decentralized versions clearly provide a better performance compared to the panmictic ones. The new mutation operator demonstrates a faster convergence on most of the studied problems versus a classical operator taken from the DE literature. Finally, the new heterogeneous decentralized DE is shown to improve the previously obtained results, and outperform the compared state-of-the-art DEs.

[1]  Swagatam Das,et al.  A closed loop stability analysis and parameter selection of the Particle Swarm Optimization dynamics for faster convergence , 2007, 2007 IEEE Congress on Evolutionary Computation.

[2]  Margaret J. Eppstein,et al.  The influence of scaling and assortativity on takeover times in scale-free topologies , 2008, GECCO '08.

[3]  Enrique Alba,et al.  Parallel Evolutionary Computations (Studies in Computational Intelligence) , 2006 .

[4]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[5]  Xing Zhong,et al.  Hierarchical Differential Evolution for Parameter Estimation in Chemical Kinetics , 2008, PRICAI.

[6]  Ruhul A. Sarker,et al.  The Self-Organization of Interaction Networks for Nature-Inspired Optimization , 2008, IEEE Transactions on Evolutionary Computation.

[7]  Ivanoe De Falco,et al.  A Distributed Differential Evolution Approach for Mapping in a Grid Environment , 2007, 15th EUROMICRO International Conference on Parallel, Distributed and Network-Based Processing (PDP'07).

[8]  Alan Godoy,et al.  A Complex Neighborhood based Particle Swarm Optimization , 2009, 2009 IEEE Congress on Evolutionary Computation.

[9]  Margaret J. Eppstein,et al.  Emergent mating topologies in spatially structured genetic algorithms , 2006, GECCO.

[10]  Francisco Luna,et al.  Advances in parallel heterogeneous genetic algorithms for continuous optimization , 2004 .

[11]  Andries Petrus Engelbrecht,et al.  Bare bones differential evolution , 2009, Eur. J. Oper. Res..

[12]  Uday K. Chakraborty,et al.  Advances in Differential Evolution , 2010 .

[13]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[14]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[15]  Fabrice Heitz,et al.  A massively parallel approach to deformable matching of 3D medical images via stochastic differential equations , 2005, Parallel Comput..

[16]  Francisco Herrera,et al.  Real-Coded Memetic Algorithms with Crossover Hill-Climbing , 2004, Evolutionary Computation.

[17]  Chih-Chin Lai,et al.  Unsupervised Clustering by Means of Hierarchical Differential Evolution Algorithm , 2008, 2008 Eighth International Conference on Intelligent Systems Design and Applications.

[18]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[19]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[20]  Marco Tomassini,et al.  Takeover time curves in random and small-world structured populations , 2005, GECCO '05.

[21]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[22]  Vitaliy Feoktistov,et al.  Differential Evolution: In Search of Solutions (Springer Optimization and Its Applications) , 2006 .

[23]  Tetsuyuki Takahama,et al.  Structural learning of neural networks by differential evolution with degeneration using mappings , 2007, 2007 IEEE Congress on Evolutionary Computation.

[24]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[25]  K. Bandurski,et al.  A parallel differential evolution algorithm for neural network training , 2006, International Symposium on Parallel Computing in Electrical Engineering (PARELEC'06).

[26]  Ville Tirronen,et al.  Distributed differential evolution with explorative–exploitative population families , 2009, Genetic Programming and Evolvable Machines.

[27]  Joachim Stender,et al.  Parallel Genetic Algorithms: Theory and Applications , 1993 .

[28]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[29]  Suganthan [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[30]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[31]  J. Kennedy Stereotyping: improving particle swarm performance with cluster analysis , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[32]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[33]  Satish Kumar,et al.  Parallel Evolutionary Asymmetric Subsethood Product Fuzzy-Neural Inference System: An Island Model Approach , 2007, 2007 International Conference on Computing: Theory and Applications (ICCTA'07).

[34]  Dario Izzo,et al.  Parallel global optimisation meta-heuristics using an asynchronous island-model , 2009, 2009 IEEE Congress on Evolutionary Computation.

[35]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[36]  Hassan M. Emara,et al.  Clubs-based Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[37]  Martin Middendorf,et al.  A hierarchical particle swarm optimizer and its adaptive variant , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[39]  Satish Kumar,et al.  Parallel Evolutionary Asymmetric Subsethood Product Fuzzy-Neural Inference System with Applications , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[40]  Enrique Alba,et al.  Improving flexibility and efficiency by adding parallelism to genetic algorithms , 2002, Stat. Comput..

[41]  M. S. Ntipteni,et al.  An Asynchronous Parallel Differential Evolution Algorithm , 2006 .

[42]  Vitaliy Feoktistov Differential Evolution: In Search of Solutions , 2006 .

[43]  A. F. Ioffe,et al.  NEW MIGRATION SCHEME FOR PARALLEL DIFFERENTIAL EVOLUTION , 2006 .

[44]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[45]  Amit Konar,et al.  Improving particle swarm optimization with differentially perturbed velocity , 2005, GECCO '05.

[46]  Amit Konar,et al.  Modeling and Analysis of the Population-Dynamics of Differential Evolution Algorithm , 2009 .

[47]  Enrique Alba,et al.  Hierarchical Cellular Genetic Algorithm , 2006, EvoCOP.

[48]  Amit Konar,et al.  Metaheuristic Clustering , 2009, Studies in Computational Intelligence.

[49]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[50]  Yan Zhou,et al.  A memetic co-evolutionary differential evolution algorithm for constrained optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[51]  Dimitris K. Tasoulis,et al.  Parallel differential evolution , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[52]  Enrique Alba,et al.  Parallel Metaheuristics: A New Class of Algorithms , 2005 .

[53]  Ivanoe De Falco,et al.  Satellite Image Registration by Distributed Differential Evolution , 2007, EvoWorkshops.

[54]  Marco Tomassini,et al.  Effects of Scale-Free and Small-World Topologies on Binary Coded Self-adaptive CEA , 2006, EvoCOP.

[55]  Surya B. Yadav,et al.  The Development and Evaluation of an Improved Genetic Algorithm Based on Migration and Artificial Selection , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[56]  Francisco Herrera,et al.  Gradual distributed real-coded genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[57]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[58]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[59]  E. Talbi Parallel combinatorial optimization , 2006 .

[60]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[61]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[62]  Enrique Alba,et al.  A Study of Canonical GAs for NSOPs , 2007, Metaheuristics.

[63]  Marco Tomassini,et al.  Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series) , 2005 .

[64]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[65]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[66]  M. Clerc,et al.  Particle Swarm Optimization , 2006 .

[67]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[68]  Ling Wang,et al.  An effective co-evolutionary differential evolution for constrained optimization , 2007, Appl. Math. Comput..

[69]  Michel Salomon,et al.  Parallélisation de l'évolution différentielle pour le recalage rigide d'images médicales volumiques , 2001 .

[70]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[71]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[72]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[73]  Hongfei Teng,et al.  Cooperative Co-evolutionary Differential Evolution for Function Optimization , 2005, ICNC.

[74]  Xiaodong Li,et al.  Adaptively Choosing Neighbourhood Bests Using Species in a Particle Swarm Optimizer for Multimodal Function Optimization , 2004, GECCO.

[75]  J. Kennedy,et al.  Neighborhood topologies in fully informed and best-of-neighborhood particle swarms , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[76]  G. Leguizamon,et al.  Island Based Distributed Differential Evolution: An Experimental Study on Hybrid Testbeds , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.