Artificial neural network regression as a local search heuristic for ensemble strategies in differential evolution

Nature frequently serves as an inspiration for developing new algorithms to solve challenging real-world problems. Mathematical modeling has led to the development of artificial neural networks (ANNs), which have proven especially useful for solving problems such as classification and regression. Moreover, evolutionary algorithms (EAs), inspired by Darwin’s natural evolution, have been successfully applied to solve optimization, modeling, and simulation problems. Differential evolution (DE) is a particularly well-known EA that possesses a multitude of strategies for generating an offspring solution, where the best strategy is not known in advance. In this paper, the ANN regression is applied as a local search heuristic within the DE algorithm that tries predicting the best strategy or attempting to generate a better offspring from an ensemble of DE strategies. This local search heuristic is applied to the population of solutions according to a control parameter that regulates between the time complexity of calculation and the quality of the solution. The experiments on a CEC 2014 test suite consisting of 30 benchmark functions reveal the full potential in developing this idea.

[1]  Carlos Gershenson,et al.  Artificial Neural Networks for Beginners , 2003, ArXiv.

[2]  Ponnuthurai N. Suganthan,et al.  Ensemble differential evolution algorithm for CEC2011 problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[3]  Ponnuthurai N. Suganthan,et al.  Differential Evolution Algorithm with Ensemble of Parameters and Mutation and Crossover Strategies , 2010, SEMCCO.

[4]  Bruce A. Robinson,et al.  Self-Adaptive Multimethod Search for Global Optimization in Real-Parameter Spaces , 2009, IEEE Transactions on Evolutionary Computation.

[5]  Josiah Adeyemo,et al.  Differential evolution algorithm for solving multi-objective crop planning model. , 2010 .

[6]  Amit Konar,et al.  Differential Evolution with Local Neighborhood , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[7]  Janez Brest,et al.  A comprehensive review of firefly algorithms , 2013, Swarm Evol. Comput..

[8]  Iztok Fister,et al.  A hybrid bat algorithm , 2013, ArXiv.

[9]  Josef Tvrdík,et al.  Differential evolution with competitive setting of control parameters , 2007 .

[10]  David Naso,et al.  Compact Differential Evolution , 2011, IEEE Transactions on Evolutionary Computation.

[11]  Bernard Widrow,et al.  Neural networks: applications in industry, business and science , 1994, CACM.

[12]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[13]  Iztok Fister,et al.  Differential evolution strategies with random forest regression in the bat algorithm , 2013, GECCO '13 Companion.

[14]  Ponnuthurai N. Suganthan,et al.  Ensemble strategies with adaptive evolutionary programming , 2010, Inf. Sci..

[15]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[16]  J. Santos,et al.  Differential Evolution for Protein Structure Prediction Using the HP Model , 2011, IWINAC.

[17]  Dilip Datta,et al.  A binary-real-coded differential evolution for unit commitment problem , 2012 .

[18]  Ahmed F. Zobaa,et al.  Neural Network Applications in Electrical Engineering , 2007, Neurocomputing.

[19]  Chin-Teng Lin,et al.  Dynamic group-based differential evolution using a self-adaptive strategy for global optimization problems , 2012, Applied Intelligence.

[20]  P. Rocca,et al.  Differential Evolution as Applied to Electromagnetics , 2011, IEEE Antennas and Propagation Magazine.

[21]  Ponnuthurai Nagaratnam Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2014 Special Session and Competition on Single Objective Real-Parameter Numerical Optimization , 2014 .

[22]  J. Tvrdík,et al.  COMPETITIVE DIFFERENTIAL EVOLUTION , 2006 .

[23]  James H. Garrett,et al.  Artificial Neural Networks for Civil Engineers: Fundamentals and Applications , 1997 .

[24]  Jouni Lampinen,et al.  A Trigonometric Mutation Operation to Differential Evolution , 2003, J. Glob. Optim..

[25]  Adam P. Piotrowski,et al.  Adaptive Memetic Differential Evolution with Global and Local neighborhood-based mutation operators , 2013, Inf. Sci..

[26]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[27]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[28]  Ruhul A. Sarker,et al.  Multi-operator based evolutionary algorithms for solving constrained optimization problems , 2011, Comput. Oper. Res..

[29]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[30]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[31]  A. Kai Qin,et al.  Self-adaptive differential evolution algorithm for numerical optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[32]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[33]  P. K. Chattopadhyay,et al.  Solving economic emission load dispatch problems using hybrid differential evolution , 2011, Appl. Soft Comput..

[34]  Belgin Emre Turkay,et al.  A novel differential evolution application to short-term electrical power generation scheduling , 2011 .

[35]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[36]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[37]  Ferrante Neri,et al.  Memetic Compact Differential Evolution for Cartesian Robot Control , 2010, IEEE Computational Intelligence Magazine.

[38]  Goran Turk,et al.  Fire Analysis of Steel Frames with the use of Artificial Neural Networks , 2007 .

[39]  Josef Tvrdík.  Adaptation in differential evolution: A numerical comparison , 2009, Appl. Soft Comput..

[40]  Kao-Shing Hwang,et al.  CO-EVOLUTIONARY HYBRID DIFFERENTIAL EVOLUTION FOR MIXED-INTEGER OPTIMIZATION PROBLEMS , 2001 .

[41]  Giovanni Iacca,et al.  Disturbed Exploitation compact Differential Evolution for limited memory optimization problems , 2011, Inf. Sci..

[42]  Joseph P. Bigus,et al.  Data mining with neural networks: solving business problems from application development to decision support , 1996 .

[43]  Xiao-Feng Xie,et al.  DEPSO: hybrid particle swarm with differential evolution operator , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[44]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[45]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[46]  N. Pierce.  Origin of Species , 1914, Nature.

[47]  Iztok Fister,et al.  Planning the sports training sessions with the bat algorithm , 2015, Neurocomputing.

[48]  Antonio LaTorre,et al.  A MOS-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test , 2011, Soft Comput..

[49]  Nikolaos Kourentzes,et al.  Neural network ensemble operators for time series forecasting , 2014, Expert Syst. Appl..

[50]  P. Suganthan,et al.  Differential evolution algorithm with ensemble of populations for global numerical optimization , 2009 .

[51]  Sangjae Lee,et al.  Predicting the helpfulness of online reviews using multilayer perceptron neural networks , 2014, Expert Syst. Appl..

[52]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[53]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[54]  Juan Humberto Sossa Azuela,et al.  Design of Artificial Neural Networks Using Differential Evolution Algorithm , 2010, ICONIP.

[55]  Yiqiao Cai,et al.  Differential Evolution With Neighborhood and Direction Information for Numerical Optimization , 2013, IEEE Transactions on Cybernetics.