Algorithmic design issues in adaptive differential evolution schemes: Review and taxonomy

Abstract The performance of most metaheuristic algorithms depends on parameters whose settings essentially serve as a key function in determining the quality of the solution and the efficiency of the search. A trend that has emerged recently is to make the algorithm parameters automatically adapt to different problems during optimization, thereby liberating the user from the tedious and time-consuming task of manual setting. These fine-tuning techniques continue to be the object of ongoing research. Differential evolution (DE) is a simple yet powerful population-based metaheuristic. It has demonstrated good convergence, and its principles are easy to understand. DE is very sensitive to its parameter settings and mutation strategy; thus, this study aims to investigate these settings with the diverse versions of adaptive DE algorithms. This study has two main objectives: (1) to present an extension for the original taxonomy of evolutionary algorithms (EAs) parameter settings that has been overlooked by prior research and therefore minimize any confusion that might arise from the former taxonomy and (2) to investigate the various algorithmic design schemes that have been used in the different variants of adaptive DE and convey them in a new classification style. In other words, this study describes in depth the structural analysis and working principle that underlie the promising and recent work in this field, to analyze their advantages and disadvantages and to gain future insights that can further improve these algorithms. Finally, the interpretation of the literature and the comparative analysis of the algorithmic schemes offer several guidelines for designing and implementing adaptive DE algorithms. The proposed design framework provides readers with the main steps required to integrate any proposed meta-algorithm into parameter and/or strategy adaptation schemes.

[1]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[2]  Ponnuthurai N. Suganthan,et al.  Ensemble and Arithmetic Recombination-Based Speciation Differential Evolution for Multimodal Optimization , 2016, IEEE Transactions on Cybernetics.

[3]  Jouni Lampinen,et al.  A Trigonometric Mutation Operation to Differential Evolution , 2003, J. Glob. Optim..

[4]  Ning Xiong,et al.  Adapting Differential Evolution Algorithms For Continuous Optimization Via Greedy Adjustment Of Control Parameters , 2016, J. Artif. Intell. Soft Comput. Res..

[5]  Janez Brest,et al.  A Brief Review of Nature-Inspired Algorithms for Optimization , 2013, ArXiv.

[6]  M.A. Vega-Rodriguez,et al.  Population-Based Incremental Learning to Solve the FAP Problem , 2008, 2008 The Second International Conference on Advanced Engineering Computing and Applications in Sciences.

[7]  Chaohua Dai,et al.  Dynamic multi-group self-adaptive differential evolution algorithm for reactive power optimization , 2010 .

[8]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[9]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[10]  Janez Brest,et al.  Population Reduction Differential Evolution with Multiple Mutation Strategies in Real World Industry Challenges , 2012, ICAISC.

[11]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[12]  Yang Tang,et al.  Adaptive population tuning scheme for differential evolution , 2013, Inf. Sci..

[13]  Radka Polakova,et al.  L-SHADE with competing strategies applied to constrained optimization , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[14]  Christian Gagné,et al.  Improving genetic algorithms performance via deterministic population shrinkage , 2009, GECCO.

[15]  Janez Brest,et al.  Structured Population Size Reduction Differential Evolution with Multiple Mutation Strategies on CEC 2013 real parameter optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[16]  Ponnuthurai N. Suganthan,et al.  An Adaptive Differential Evolution Algorithm With Novel Mutation and Crossover Strategies for Global Numerical Optimization , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Haifeng Li,et al.  Ensemble of differential evolution variants , 2018, Inf. Sci..

[18]  Qingfu Zhang,et al.  Differential Evolution With Composite Trial Vector Generation Strategies and Control Parameters , 2011, IEEE Transactions on Evolutionary Computation.

[19]  Ilpo Poikolainen,et al.  Micro-differential evolution with extra moves along the axes , 2013, 2013 IEEE Symposium on Differential Evolution (SDE).

[20]  Orhan Arikan,et al.  Self-adaptive randomized and rank-based differential evolution for multimodal problems , 2011, J. Glob. Optim..

[21]  Janez Brest,et al.  An improved self-adaptive differential evolution algorithm in single objective constrained real-parameter optimization , 2010, IEEE Congress on Evolutionary Computation.

[22]  Qingfu Zhang,et al.  Enhancing the search ability of differential evolution through orthogonal crossover , 2012, Inf. Sci..

[23]  Xuefeng Yan,et al.  Self-Adaptive Differential Evolution Algorithm With Zoning Evolution of Control Parameters and Adaptive Mutation Strategies , 2016, IEEE Transactions on Cybernetics.

[24]  Mark Hoogendoorn,et al.  Parameter Control in Evolutionary Algorithms: Trends and Challenges , 2015, IEEE Transactions on Evolutionary Computation.

[25]  M. Montaz Ali,et al.  Population set-based global optimization algorithms: some modifications and numerical studies , 2004, Comput. Oper. Res..

[26]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[27]  Janez Brest,et al.  Performance comparison of self-adaptive and adaptive differential evolution algorithms , 2007, Soft Comput..

[28]  Quanyuan Feng,et al.  A comparative study of crossover in differential evolution , 2011, J. Heuristics.

[29]  Ali Wagdy Mohamed,et al.  Real-parameter unconstrained optimization based on enhanced fitness-adaptive differential evolution algorithm with novel mutation , 2017, Soft Computing.

[30]  Janez Brest,et al.  Self-adaptive control parameters' randomization frequency and propagations in differential evolution , 2015, Swarm Evol. Comput..

[31]  Chun Chen,et al.  Multiple trajectory search for Large Scale Global Optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[32]  Zbigniew Michalewicz,et al.  Parameter control in evolutionary algorithms , 1999, IEEE Trans. Evol. Comput..

[33]  Petr Bujok,et al.  Differential evolution with rotation-invariant mutation and competing-strategies adaptation , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[34]  Hui Li,et al.  Enhanced Differential Evolution With Adaptive Strategies for Numerical Optimization , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Saku Kukkonen,et al.  Real-parameter optimization with differential evolution , 2005, 2005 IEEE Congress on Evolutionary Computation.

[36]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[37]  H. Abbass The self-adaptive Pareto differential evolution algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[38]  Patrick Siarry,et al.  A survey on optimization metaheuristics , 2013, Inf. Sci..

[39]  Ali Wagdy Mohamed,et al.  Adaptive guided differential evolution algorithm with novel mutation for numerical optimization , 2017, International Journal of Machine Learning and Cybernetics.

[40]  Mariane R. Petraglia,et al.  Global Optimization and Its Applications , 2012 .

[41]  Pratyusha Rakshit,et al.  Differential evolution induced many objective optimization , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[42]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[43]  Maoguo Gong,et al.  Metaheuristic Optimization: Algorithmic Design and Applications , 2017 .

[44]  Swagatam Das,et al.  An improved differential evolution algorithm with fitness-based adaptation of the control parameters , 2011, Inf. Sci..

[45]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[46]  Robert E. Smith,et al.  Adaptively Resizing Populations: An Algorithm and Analysis , 1993, ICGA.

[47]  Valder Steffen,et al.  Self-adaptive Multi-objective Optimization Differential Evolution , 2017 .

[48]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[49]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[50]  Millie Pant,et al.  An efficient Differential Evolution based algorithm for solving multi-objective optimization problems , 2011, Eur. J. Oper. Res..

[51]  Zhijian Wu,et al.  Parallel differential evolution with self-adapting control parameters and generalized opposition-based learning for solving high-dimensional optimization problems , 2013, J. Parallel Distributed Comput..

[52]  Ferrante Neri,et al.  Optimization of Delayed-State Kalman-Filter-Based Algorithm via Differential Evolution for Sensorless Control of Induction Motors , 2010, IEEE Transactions on Industrial Electronics.

[53]  Stefan Janaqi,et al.  New Strategies in Differential Evolution , 2004 .

[54]  Efrén Mezura-Montes,et al.  An experimental comparison of two constraint handling approaches used with differential evolution , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[55]  Rainer Storn Real-world applications in the communications industry - when do we resort to Differential Evolution? , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[56]  Ruhul A. Sarker,et al.  A self-adaptive combined strategies algorithm for constrained optimization using differential evolution , 2014, Appl. Math. Comput..

[57]  Jason Teo,et al.  Exploring dynamic self-adaptive populations in differential evolution , 2006, Soft Comput..

[58]  Carlos A. Coello Coello,et al.  Simple Feasibility Rules and Differential Evolution for Constrained Optimization , 2004, MICAI.

[59]  Rammohan Mallipeddi Harmony Search Based Parameter Ensemble Adaptation for Differential Evolution , 2013, J. Appl. Math..

[60]  Carlos Cruz Corona,et al.  Self-adaptive, multipopulation differential evolution in dynamic environments , 2013, Soft Comput..

[61]  J. Lampinen A constraint handling approach for the differential evolution algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[62]  Ville Tirronen,et al.  Shuffle or update parallel differential evolution for large-scale optimization , 2011, Soft Comput..

[63]  Michal Pluhacek,et al.  Archive Analysis in SHADE , 2017, ICAISC.

[64]  Ales Zamuda,et al.  Adaptive constraint handling and Success History Differential Evolution for CEC 2017 Constrained Real-Parameter Optimization , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[65]  Ferrante Neri,et al.  Differential Evolution with Noise Analyzer , 2009, EvoWorkshops.

[66]  Carlos A. Coello Coello,et al.  Multi-objective Optimization Using Differential Evolution: A Survey of the State-of-the-Art , 2008 .

[67]  Bijaya K. Panigrahi,et al.  A noise resilient Differential Evolution with improved parameter and strategy control , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[68]  Ville Tirronen,et al.  On memetic Differential Evolution frameworks: A study of advantages and limitations in hybridization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[69]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[70]  Ali Wagdy Mohamed,et al.  Solving large-scale global optimization problems using enhanced adaptive differential evolution algorithm , 2017 .

[71]  Tapabrata Ray,et al.  Adaptation of operators and continuous control parameters in differential evolution for constrained optimization , 2017, Soft Computing.

[72]  Ponnuthurai N. Suganthan,et al.  Ensemble of parameters in a sinusoidal differential evolution with niching-based population reduction , 2017, Swarm Evol. Comput..

[73]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[74]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[75]  Bogdan Filipic,et al.  DEMO: Differential Evolution for Multiobjective Optimization , 2005, EMO.

[76]  Josef Tvrdík Adaptation in differential evolution: A numerical comparison , 2009, Appl. Soft Comput..

[77]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[78]  Swagatam Das,et al.  A Cluster-Based Differential Evolution Algorithm With External Archive for Optimization in Dynamic Environments , 2013, IEEE Transactions on Cybernetics.

[79]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[80]  Sankha Subhra Mullick,et al.  A Switched Parameter Differential Evolution for Large Scale Global Optimization - Simpler May Be Better , 2015, MENDEL.

[81]  Efrén Mezura-Montes,et al.  Differential evolution in constrained numerical optimization: An empirical study , 2010, Inf. Sci..

[82]  Swagatam Das,et al.  An Adaptive Differential Evolution Algorithm for Global Optimization in Dynamic Environments , 2014, IEEE Transactions on Cybernetics.

[83]  Robert G. Reynolds,et al.  An ensemble sinusoidal parameter adaptation incorporated with L-SHADE for solving CEC2014 benchmark problems , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[84]  Stefan Janaqi,et al.  Generalization of the strategies in differential evolution , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[85]  Janez Brest,et al.  Dynamic optimization using Self-Adaptive Differential Evolution , 2009, 2009 IEEE Congress on Evolutionary Computation.

[86]  Efrén Mezura-Montes,et al.  The baldwin effect on a memetic differential evolution for constrained numerical optimization problems , 2017, GECCO.

[87]  Alex S. Fukunaga,et al.  Evaluating the performance of SHADE on CEC 2013 benchmark problems , 2013, 2013 IEEE Congress on Evolutionary Computation.

[88]  Xiangtao Li,et al.  Multi-search differential evolution algorithm , 2017, Applied Intelligence.

[89]  Saad Mekhilef,et al.  Parameters' fine tuning of differential evolution algorithm , 2015, Comput. Syst. Sci. Eng..

[90]  Petr Bujok,et al.  Enhanced individual-dependent differential evolution with population size adaptation , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[91]  Jing Xiao,et al.  Classification-based self-adaptive differential evolution with fast and reliable convergence performance , 2011, Soft Comput..

[92]  Fei Peng,et al.  Multi-start JADE with knowledge transfer for numerical optimization , 2009, IEEE Congress on Evolutionary Computation.

[93]  Janez Brest,et al.  Self-adaptive differential evolution algorithm using population size reduction and three strategies , 2011, Soft Comput..

[94]  Alex S. Fukunaga,et al.  Improving the search performance of SHADE using linear population size reduction , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[95]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[96]  Robert G. Reynolds,et al.  A novel differential crossover strategy based on covariance matrix learning with Euclidean neighborhood for solving real-world problems , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[97]  Ferrante Neri,et al.  Memetic Compact Differential Evolution for Cartesian Robot Control , 2010, IEEE Computational Intelligence Magazine.

[98]  Ali Wagdy Mohamed,et al.  A novel differential evolution algorithm for solving constrained engineering optimization problems , 2017, Journal of Intelligent Manufacturing.

[99]  Anas A. Hadi,et al.  LSHADE with semi-parameter adaptation hybrid with CMA-ES for solving CEC 2017 benchmark problems , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[100]  M. Al-Dabbagh,et al.  A new modified differential evolution algorithm scheme-based linear frequency modulation radar signal de-noising , 2015 .

[101]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[102]  Ning Ma,et al.  Fast 3D path planning based on heuristic-aided differential evolution , 2017, GECCO.

[103]  Václav Snásel,et al.  Utilization of the Discrete Differential Evolution for Optimization in Multidimensional Point Clouds , 2016, Comput. Intell. Neurosci..

[104]  E. Bor,et al.  Differential evolution algorithm based photonic structure design: numerical and experimental verification of subwavelength λ/5 focusing of light , 2016, Scientific Reports.

[105]  Xianpeng Wang,et al.  A multi-objective differential evolution algorithm with memory based population construction , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[106]  Chun Chen,et al.  Multiple trajectory search for multiobjective optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[107]  Kaisa Miettinen,et al.  A new hybrid mutation operator for multiobjective optimization with differential evolution , 2011, Soft Comput..

[108]  Ponnuthurai N. Suganthan,et al.  Differential Evolution with Two Subpopulations , 2014, SEMCCO.

[109]  Rainer Storn,et al.  Minimizing the real functions of the ICEC'96 contest by differential evolution , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[110]  Robert E. Smith,et al.  Adaptively Resizing Populations: Algorithm, Analysis, and First Results , 1993, Complex Syst..

[111]  Dexuan Zou,et al.  A novel modified differential evolution algorithm for constrained optimization problems , 2011, Comput. Math. Appl..

[112]  Millie Pant,et al.  Application of differential evolution algorithm for optimization of strategies based on financial time series , 2016 .

[113]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[114]  Kenneth V. Price,et al.  An introduction to differential evolution , 1999 .

[115]  David B. Fogel,et al.  Meta-evolutionary programming , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[116]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[117]  Zhihua Cui,et al.  Swarm Intelligence and Bio-Inspired Computation: Theory and Applications , 2013 .

[118]  David Naso,et al.  Compact Differential Evolution , 2011, IEEE Transactions on Evolutionary Computation.

[119]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[120]  Tapabrata Ray,et al.  Differential Evolution With Dynamic Parameters Selection for Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[121]  Luc Martens,et al.  Optimization of Power Consumption in 4G LTE Networks Using a Novel Barebones Self-adaptive Differential Evolution Algorithm , 2017, Telecommun. Syst..

[122]  Ponnuthurai N. Suganthan,et al.  Self-adaptive differential evolution with multi-trajectory search for large-scale optimization , 2011, Soft Comput..

[123]  Gu Yu An improved differential evolution algorithm for TSP , 2011 .

[124]  Y. Wang,et al.  An empirical study of multifactorial PSO and multifactorial DE , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[125]  Ponnuthurai N. Suganthan,et al.  Ensemble sinusoidal differential covariance matrix adaptation with Euclidean neighborhood for solving CEC2017 benchmark problems , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[126]  Jouni Lampinen,et al.  A Fuzzy Adaptive Differential Evolution Algorithm , 2005, Soft Comput..

[127]  Liang Chen,et al.  A Novel Self-adaptive Differential Evolution Algorithm with Population Size Adjustment Scheme , 2014 .

[128]  Ali Wagdy Mohamed,et al.  Constrained optimization based on modified differential evolution algorithm , 2012, Inf. Sci..

[129]  Ali Wagdy Mohamed,et al.  An improved differential evolution algorithm with triangular mutation for global numerical optimization , 2015, Comput. Ind. Eng..

[130]  Ville Tirronen,et al.  Scale factor inheritance mechanism in distributed differential evolution , 2009, Soft Comput..

[131]  A. Kai Qin,et al.  Self-adaptive differential evolution algorithm for numerical optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[132]  Hammoudi Abderazek,et al.  Adaptive mixed differential evolution algorithm for bi-objective tooth profile spur gear optimization , 2017 .