Differential Evolution algorithm with Separated Groups for multi-dimensional optimization problems

The classical Differential Evolution (DE) algorithm, one of population-based Evolutionary Computation methods, proved to be a successful approach for relatively simple problems, but does not perform well for difficult multi-dimensional non-convex functions. A number of significant modifications of DE have been proposed in recent years, including very few approaches referring to the idea of distributed Evolutionary Algorithms. The present paper presents a new algorithm to improve optimization performance, namely DE with Separated Groups (DE-SG), which distributes population into small groups, defines rules of exchange of information and individuals between the groups and uses two different strategies to keep balance between exploration and exploitation capabilities. The performance of DE-SG is compared to that of eight algorithms belonging to the class of Evolutionary Strategies (Covariance Matrix Adaptation ES), Particle Swarm Optimization (Comprehensive Learning PSO and Efficient Population Utilization Strategy PSO), Differential Evolution (Distributed DE with explorative-exploitative population families, Self-adaptive DE, DE with global and local neighbours and Grouping Differential Evolution) and multi-algorithms (AMALGAM). The comparison is carried out for a set of 10-, 30- and 50-dimensional rotated test problems of varying difficulty, including 10- and 30-dimensional composition functions from CEC2005. Although slow for simple functions, the proposed DE-SG algorithm achieves a great success rate for more difficult 30- and 50-dimensional problems.

[1]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[2]  Andries Petrus Engelbrecht,et al.  Empirical analysis of self-adaptive differential evolution , 2007, Eur. J. Oper. Res..

[3]  M. M. Ali,et al.  A numerical study of some modified differential evolution algorithms , 2006, Eur. J. Oper. Res..

[4]  Adam P. Piotrowski,et al.  The grouping differential evolution algorithm for multi-dimensional optimization problems , 2010 .

[5]  G. Leguizamon,et al.  Island Based Distributed Differential Evolution: An Experimental Study on Hybrid Testbeds , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[6]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[7]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[8]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[9]  Malcolm J. Beynon,et al.  Evidence-based modelling of strategic fit: An introduction to RCaRBS , 2010, Eur. J. Oper. Res..

[10]  Janez Brest,et al.  Performance comparison of self-adaptive and adaptive differential evolution algorithms , 2007, Soft Comput..

[11]  R. Salomon Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[12]  Adam P. Piotrowski,et al.  Evaluation of temporal concentration profiles for ungauged rivers following pollution incidents , 2011 .

[13]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[14]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[15]  Bruce A. Robinson,et al.  Self-Adaptive Multimethod Search for Global Optimization in Real-Parameter Spaces , 2009, IEEE Transactions on Evolutionary Computation.

[16]  L. Darrell Whitley,et al.  Evaluating Evolutionary Algorithms , 1996, Artif. Intell..

[17]  Ville Tirronen,et al.  Distributed differential evolution with explorative–exploitative population families , 2009, Genetic Programming and Evolvable Machines.

[18]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[19]  Andries Petrus Engelbrecht,et al.  Bare bones differential evolution , 2009, Eur. J. Oper. Res..

[20]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[21]  Riccardo Poli,et al.  Evolving Problems to Learn About Particle Swarm Optimizers and Other Search Algorithms , 2006, IEEE Transactions on Evolutionary Computation.

[22]  Andries Petrus Engelbrecht,et al.  Self-adaptive Differential Evolution , 2005, CIS.

[23]  SK Mishra,et al.  Global Optimization by Differential Evolution and Particle Swarm Methods: Evaluation on Some Benchmark Functions , 2006 .

[24]  Masao Fukushima,et al.  Tabu Search directed by direct search methods for nonlinear global optimization , 2006, Eur. J. Oper. Res..

[25]  Amit Konar,et al.  Particle Swarm Optimization and Differential Evolution Algorithms: Technical Analysis, Applications and Hybridization Perspectives , 2008, Advances of Computational Intelligence in Industrial Systems.

[26]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[27]  Joni-Kristian Kämäräinen,et al.  Differential Evolution Training Algorithm for Feed-Forward Neural Networks , 2003, Neural Processing Letters.

[28]  Jingqiao Zhang,et al.  Evolutionary optimization of transition probability matrices for credit decision-making , 2010, Eur. J. Oper. Res..

[29]  A. Kai Qin,et al.  Self-adaptive differential evolution algorithm for numerical optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[30]  Ernesto P. Adorio,et al.  MVF - Multivariate Test Functions Library in C for Unconstrained Global Optimization , 2005 .

[31]  Shang-Jeng Tsai,et al.  Efficient Population Utilization Strategy for Particle Swarm Optimizer , 2009, IEEE Trans. Syst. Man Cybern. Part B.

[32]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[33]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[34]  Enrique Alba,et al.  Benchmarking a Wide Spectrum of Metaheuristic Techniques for the Radio Network Design Problem , 2009, IEEE Transactions on Evolutionary Computation.

[35]  John H. Holland,et al.  Building Blocks, Cohort Genetic Algorithms, and Hyperplane-Defined Functions , 2000, Evolutionary Computation.

[36]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[37]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[38]  Amit Konar,et al.  Differential Evolution with Local Neighborhood , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[39]  Dana Petcu,et al.  Adaptive Pareto Differential Evolution and Its Parallelization , 2003, PPAM.

[40]  Tom Van Woensel,et al.  On the system optimum of traffic assignment in M/G/c/c state-dependent queueing networks , 2010, Eur. J. Oper. Res..

[41]  Carlos A. Coello Coello,et al.  A comparative study of differential evolution variants for global optimization , 2006, GECCO.

[42]  Ivanoe De Falco,et al.  Distributed Differential Evolution for the Registration of Remotely Sensed Images , 2007, 15th EUROMICRO International Conference on Parallel, Distributed and Network-Based Processing (PDP'07).

[43]  Marco Tomassini,et al.  Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series) , 2005 .

[44]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[45]  M. Clerc,et al.  Particle Swarm Optimization , 2006 .

[46]  Edmund K. Burke,et al.  The Speciating Island Model: An alternative parallel evolutionary algorithm , 2006, J. Parallel Distributed Comput..

[47]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[48]  Dimitris K. Tasoulis,et al.  Parallel differential evolution , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[49]  Fawaz S. Al-Anzi,et al.  A self-adaptive differential evolution heuristic for two-stage assembly scheduling problem to minimize maximum lateness with setup times , 2007, Eur. J. Oper. Res..

[50]  Saku Kukkonen,et al.  Real-parameter optimization with differential evolution , 2005, 2005 IEEE Congress on Evolutionary Computation.

[51]  L. Darrell Whitley,et al.  Ruffled by Ridges: How Evolutionary Algorithms Can Fail , 2004, GECCO.

[52]  Zelda B. Zabinsky,et al.  A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems , 2005, J. Glob. Optim..

[53]  Jouni Lampinen,et al.  A Fuzzy Adaptive Differential Evolution Algorithm , 2005, Soft Comput..

[54]  Xiao-Feng Xie,et al.  DEPSO: hybrid particle swarm with differential evolution operator , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[55]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[56]  Adam P. Piotrowski,et al.  Estimation of parameters of the transient storage model by means of multi-layer perceptron neural networks / Estimation des paramètres du modèle de transport TSM au moyen de réseaux de neurones perceptrons multi-couches , 2008 .

[57]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[58]  Ralph R. Martin,et al.  A Sequential Niche Technique for Multimodal Function Optimization , 1993, Evolutionary Computation.

[59]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[60]  Adam P. Piotrowski,et al.  Optimizing neural networks for river flow forecasting – Evolutionary Computation methods versus the Levenberg–Marquardt approach , 2011 .

[61]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.