Improved Differential Evolution for Large-Scale Black-Box Optimization

The demand for solving large-scale complex problems continues to grow. Many real-world problems are described by a large number of variables that interact with each other in a complex way. The dimensionality of the problem has a direct impact on the computational cost of the optimization. During the last two decades, differential evolution has been shown to be one of the most powerful optimizers for a wide range of optimization problems. In this paper, we investigate its appropriateness for large-scale problems. We propose a new variation of differential evolution that exhibits good results on difficult functions with a large numbers of variables. The proposed algorithm incorporates the following mechanisms: the use of three strategies, the extended range of values for self-adapted parameters <inline-formula> <tex-math notation="LaTeX">$F$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\mathit {CR}$ </tex-math></inline-formula>, subpopulations, and the population size reduction. The algorithm was tested on the CEC 2013 benchmark suite for large-scale optimization, and on two real-world problems from the CEC 2011 benchmark suite on real-world optimization. A comparative analysis was performed with recently proposed algorithms. The analysis shows the superior performance of our algorithm on most complex problems, described by overlapping and non-separable functions.

[1]  Francisco Herrera,et al.  Iterative hybridization of DE with local search for the CEC'2015 special session on large scale global optimization , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[2]  Janez Brest,et al.  High-dimensional real-parameter optimization using Self-Adaptive Differential Evolution algorithm with population size reduction , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[3]  Janez Brest,et al.  Self-adaptive control parameters' randomization frequency and propagations in differential evolution , 2015, Swarm Evol. Comput..

[4]  Giuseppe A. Trunfio,et al.  A new algorithm for adapting the configuration of subcomponents in large-scale optimization with cooperative coevolution , 2016, Inf. Sci..

[5]  Juan Carlos Herrera-Lozada,et al.  Micro Differential Evolution Performance Empirical Study for High Dimensional Optimization Problems , 2013, LSSC.

[6]  Sankha Subhra Mullick,et al.  A Switched Parameter Differential Evolution for Large Scale Global Optimization - Simpler May Be Better , 2015, MENDEL.

[7]  Ilya Loshchilov,et al.  LM-CMA: An Alternative to L-BFGS for Large-Scale Black Box Optimization , 2015, Evolutionary Computation.

[8]  David E. Goldberg,et al.  Dependency Structure Matrix, Genetic Algorithms, and Effective Recombination , 2009, Evolutionary Computation.

[9]  Janez Brest,et al.  Population size reduction for the differential evolution algorithm , 2008, Applied Intelligence.

[10]  P. N. Suganthan,et al.  Problem Definitions and Evaluation Criteria for CEC 2011 Competition on Testing Evolutionary Algorithms on Real World Optimization Problems , 2011 .

[11]  Tetsuyuki Takahama,et al.  Rank-based differential evolution with multiple mutation strategies for large scale global optimization , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[12]  Haiyan Liu,et al.  Cooperative Coevolution with Formula-Based Variable Grouping for Large-Scale Global Optimization , 2018, Evolutionary Computation.

[13]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[14]  Xiaodong Li,et al.  Cooperatively Coevolving Particle Swarms for Large Scale Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[15]  Janez Brest,et al.  Large Scale Global Optimization using Differential Evolution with self-adaptation and cooperative co-evolution , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[16]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[17]  Janez Brest,et al.  Large scale global optimization using self-adaptive differential evolution algorithm , 2010, IEEE Congress on Evolutionary Computation.

[18]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[19]  Sven Leyffer,et al.  A Trust-Region Algorithm for Global Optimization , 2006, Comput. Optim. Appl..

[20]  Xiaodong Li,et al.  DG2: A Faster and More Accurate Differential Grouping for Large-Scale Black-Box Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[21]  Janez Brest,et al.  Self-adaptive differential evolution algorithm with a small and varying population size , 2012, 2012 IEEE Congress on Evolutionary Computation.

[22]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[23]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[24]  Xin Yao,et al.  Self-adaptive differential evolution with neighborhood search , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[25]  Janez Brest,et al.  Self-adaptive differential evolution algorithm using population size reduction and three strategies , 2011, Soft Comput..

[26]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[27]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[28]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[29]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[30]  Antonio Bolufé Röhler,et al.  A minimum population search hybrid for large scale global optimization , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[31]  Wei Luo,et al.  Improved Differential Evolution With a Modified Orthogonal Learning Strategy , 2017, IEEE Access.

[32]  Xiaodong Li,et al.  CBCC3 — A contribution-based cooperative co-evolutionary algorithm with improved exploration/exploitation balance , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[33]  Chun Chen,et al.  Multiple trajectory search for Large Scale Global Optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[34]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[35]  Joseph Morlier,et al.  Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method , 2018 .

[36]  Xiaodong Li,et al.  Efficient Resource Allocation in Cooperative Co-Evolution for Large-Scale Global Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[37]  Xiangtao Li,et al.  Multi-search differential evolution algorithm , 2017, Applied Intelligence.

[38]  Zhongyi Hu,et al.  A PSO and pattern search based memetic algorithm for SVMs parameters optimization , 2013, Neurocomputing.

[39]  Ponnuthurai Nagaratnam Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2014 Special Session and Competition on Single Objective Real-Parameter Numerical Optimization , 2014 .

[40]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[41]  Yanchun Liang,et al.  A cooperative particle swarm optimizer with statistical variable interdependence learning , 2012, Inf. Sci..

[42]  Xiaodong Li,et al.  Cooperative Co-Evolution With Differential Grouping for Large Scale Optimization , 2014, IEEE Transactions on Evolutionary Computation.