Enhanced differential evolution using local Lipschitz underestimate strategy for computationally expensive optimization problems

Graphical abstractDisplay Omitted HighlightsThis study presents a DE with local Lipschitz underestimated strategy to reduce the number of function evaluations.Lipschitz supporting hyperplanes are constructed based on the evolutionary information to guide the DE process.The proposed local Lipschitz underestimate strategy is general and can be applied to other evolutionary algorithms. Differential evolution (DE) has been successfully applied in many scientific and engineering fields. However, one of the main problems in using DE is that the optimization process usually needs a large number of function evaluations to find an acceptable solution, which leads to an increase of computational time, particularly in the case of computationally expensive problems. The Lipschitz underestimate method (LUM), a deterministic global optimization technique, can be used to obtain the underestimate of the objective function by building a sequence of piecewise linear supporting hyperplanes. In this paper, an enhanced differential evolution using local Lipschitz underestimate strategy, called LLUDE, is proposed for computationally expensive optimization problems. LLUDE effectively combines the exploration of DE with the underestimation of LUM to obtain promising solutions with less function evaluations. To verify the performance of LLUDE, 18 well-known benchmark functions and one computationally expensive real-world application problem, namely, the protein structure prediction problem, are employed. Results obtained from these benchmark functions show that LLUDE is significantly better than or at least comparable to the state-of-the-art DE variants and non-DE algorithms. Furthermore, the results obtained from the protein structure prediction problem suggest that LLUDE is effective and efficient for computationally expensive problems.

[1]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[2]  Alberto Ferrer,et al.  Bounded lower subdifferentiability optimization techniques: applications , 2010, J. Glob. Optim..

[3]  Ioannis Ch. Paschalidis,et al.  SDU: A Semidefinite Programming-Based Underestimation Method for Stochastic Global Optimization in Protein Docking , 2007, IEEE Transactions on Automatic Control.

[4]  Ponnuthurai Nagaratnam Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2014 Special Session and Competition on Single Objective Real-Parameter Numerical Optimization , 2014 .

[5]  Qingfu Zhang,et al.  Differential Evolution With Composite Trial Vector Generation Strategies and Control Parameters , 2011, IEEE Transactions on Evolutionary Computation.

[6]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[7]  Kimon P. Valavanis,et al.  Nonlinear Model Predictive Control With Neural Network Optimization for Autonomous Autorotation of Small Unmanned Helicopters , 2011, IEEE Transactions on Control Systems Technology.

[8]  André da Motta Salles Barreto,et al.  Using performance profiles to analyze the results of the 2006 CEC constrained optimization competition , 2010, IEEE Congress on Evolutionary Computation.

[9]  Tetsuyuki Takahama,et al.  A comparative study on kernel smoothers in Differential Evolution with estimated comparison method for reducing function evaluations , 2009, 2009 IEEE Congress on Evolutionary Computation.

[10]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[11]  Rammohan Mallipeddi,et al.  An evolving surrogate model-based differential evolution algorithm , 2015, Appl. Soft Comput..

[12]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[14]  Farrokh Mistree,et al.  Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization , 2001 .

[15]  Hoang-Anh Pham,et al.  Reduction of function evaluation in differential evolution using nearest neighbor comparison , 2015, Vietnam Journal of Computer Science.

[16]  Helio J. C. Barbosa,et al.  A multiple minima genetic algorithm for protein structure prediction , 2014, Appl. Soft Comput..

[17]  Robert Piché,et al.  Mixture surrogate models based on Dempster-Shafer theory for global optimization problems , 2011, J. Glob. Optim..

[18]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[19]  Robert Abel,et al.  Computational methods for high resolution prediction and refinement of protein structures. , 2013, Current opinion in structural biology.

[20]  Alberto Ferrer,et al.  Comparative study of RPSALG algorithm for convex semi-infinite programming , 2015, Comput. Optim. Appl..

[21]  David Baker,et al.  The dual role of fragments in fragment‐assembly methods for de novo protein structure prediction , 2012, Proteins.

[22]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[23]  K. Dill,et al.  The Protein-Folding Problem, 50 Years On , 2012, Science.

[24]  Alagan Anpalagan,et al.  Differential evolution aided adaptive resource allocation in OFDMA systems with proportional rate constraints , 2015, Appl. Soft Comput..

[25]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[26]  Samir Sayah,et al.  A hybrid differential evolution algorithm based on particle swarm optimization for nonconvex economic dispatch problems , 2013, Appl. Soft Comput..

[27]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[28]  Yang Liu,et al.  A fast differential evolution algorithm using k-Nearest Neighbour predictor , 2011, Expert Syst. Appl..

[29]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[30]  David Baker,et al.  Protein Structure Prediction Using Rosetta , 2004, Numerical Computer Methods, Part D.

[31]  Mesut Gündüz,et al.  A recombination-based hybridization of particle swarm optimization and artificial bee colony algorithm for continuous optimization problems , 2013, Appl. Soft Comput..

[32]  Ju-Jang Lee,et al.  An efficient differential evolution using speeded-up k-nearest neighbor estimator , 2014, Soft Comput..

[33]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[34]  Nur Evin Özdemirel,et al.  Ant Colony Optimization based clustering methodology , 2015, Appl. Soft Comput..

[35]  Carlos García-Martínez,et al.  Global and local real-coded genetic algorithms based on parent-centric crossover operators , 2008, Eur. J. Oper. Res..

[36]  Riccardo Poli,et al.  Evolving Problems to Learn About Particle Swarm Optimizers and Other Search Algorithms , 2006, IEEE Transactions on Evolutionary Computation.

[37]  Helio J. C. Barbosa,et al.  Full-atom ab initio protein structure prediction with a Genetic Algorithm using a similarity-based surrogate model , 2010, IEEE Congress on Evolutionary Computation.

[38]  Bin Li,et al.  A New Memetic Algorithm With Fitness Approximation for the Defect-Tolerant Logic Mapping in Crossbar-Based Nanoarchitectures , 2014, IEEE Transactions on Evolutionary Computation.

[39]  G. Beliakov Extended cutting angle method of global optimization , 2008 .

[40]  S. Baskar,et al.  Protein structure prediction using diversity controlled self-adaptive differential evolution with local search , 2015, Soft Comput..

[41]  Hui Wang,et al.  Gaussian Bare-Bones Differential Evolution , 2013, IEEE Transactions on Cybernetics.

[42]  Christodoulos A. Floudas,et al.  Advances in protein structure prediction and de novo protein design : A review , 2006 .

[43]  Pedro J. Ballester,et al.  Ultrafast shape recognition for similarity search in molecular databases , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[44]  Zhening Li,et al.  Approximation Methods for Polynomial Optimization: Models, Algorithms, and Applications , 2012 .

[45]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[46]  Adil M. Bagirov,et al.  Solving DC programs using the cutting angle method , 2015, J. Glob. Optim..

[47]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[48]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[49]  Christine A. Shoemaker,et al.  Improved Strategies for Radial basis Function Methods for Global Optimization , 2007, J. Glob. Optim..

[50]  Mahdi Taghizadeh,et al.  Colonial competitive differential evolution: An experimental study for optimal economic load dispatch , 2016, Appl. Soft Comput..

[51]  T. Vengattaraman,et al.  Gene Suppressor: An added phase toward solving large scale optimization problems in genetic algorithm , 2015, Appl. Soft Comput..

[52]  Tapabrata Ray,et al.  A surrogate-assisted differential evolution algorithm with dynamic parameters selection for solving expensive optimization problems , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[53]  Gleb Beliakov,et al.  Challenges of continuous global optimization in molecular structure prediction , 2007, Eur. J. Oper. Res..