Evolution by Adapting Surrogates

To deal with complex optimization problems plagued with computationally expensive fitness functions, the use of surrogates to replace the original functions within the evolutionary framework is becoming a common practice. However, the appropriate datacentric approximation methodology to use for the construction of surrogate model would depend largely on the nature of the problem of interest, which varies from fitness landscape and state of the evolutionary search, to the characteristics of search algorithm used. This has given rise to the plethora of surrogate-assisted evolutionary frameworks proposed in the literature with ad hoc approximation/surrogate modeling methodologies considered. Since prior knowledge on the suitability of the data centric approximation methodology to use in surrogate-assisted evolutionary optimization is typically unavailable beforehand, this paper presents a novel evolutionary framework with the evolvability learning of surrogates (EvoLS) operating on multiple diverse approximation methodologies in the search. Further, in contrast to the common use of fitness prediction error as a criterion for the selection of surrogates, the concept of evolvability to indicate the productivity or suitability of an approximation methodology that brings about fitness improvement in the evolutionary search is introduced as the basis for adaptation. The backbone of the proposed EvoLS is a statistical learning scheme to determine the evolvability of each approximation methodology while the search progresses online. For each individual solution, the most productive approximation methodology is inferred, that is, the method with highest evolvability measure. Fitness improving surrogates are subsequently constructed for use within a trust-region enabled local search strategy, leading to the self-configuration of a surrogate-assisted memetic algorithm for solving computationally expensive problems. A numerical study of EvoLS on commonly used benchmark problems and a real-world computationally expensive aerodynamic car rear design problem highlights the efficacy of the proposed EvoLS in attaining reliable, high quality, and efficient performance under a limited computational budget.

[1]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[2]  Hrvoje Jasak,et al.  Dynamic Mesh Handling in OpenFOAM , 2009 .

[3]  Yoel Tenne,et al.  A Model-Assisted Memetic Algorithm for Expensive Optimization Problems , 2009, Nature-Inspired Algorithms for Optimisation.

[4]  Salvador Pintos,et al.  An Optimization Methodology of Alkaline-Surfactant-Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates , 2004 .

[5]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[6]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[7]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[8]  Bernhard Sendhoff,et al.  Lamarckian memetic algorithms: local optimum and connectivity structure analysis , 2009, Memetic Comput..

[9]  Leslie G. Valiant,et al.  Evolvability , 2009, JACM.

[10]  John E. Dennis,et al.  A framework for managing models in nonlinear optimization of computationally expensive functions , 1999 .

[11]  Aleksandar Jemcov,et al.  OpenFOAM: A C++ Library for Complex Physics Simulations , 2007 .

[12]  Andy J. Keane,et al.  Multi-Objective Optimization Using Surrogates , 2010 .

[13]  Yoel Tenne,et al.  Metamodel accuracy assessment in evolutionary optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[14]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[15]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[16]  Yun-Wei Shang,et al.  A Note on the Extended Rosenbrock Function , 2006, Evolutionary Computation.

[17]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[18]  L. Altenberg,et al.  PERSPECTIVE: COMPLEX ADAPTATIONS AND THE EVOLUTION OF EVOLVABILITY , 1996, Evolution; international journal of organic evolution.

[19]  M. Rais-Rohani,et al.  Ensemble of metamodels with optimized weight factors , 2008 .

[20]  Michèle Sebag,et al.  Adaptive operator selection with dynamic multi-armed bandits , 2008, GECCO '08.

[21]  Carlos A. Coello Coello,et al.  A Review of Techniques for Handling Expensive Functions in Evolutionary Multi-Objective Optimization , 2010 .

[22]  Néstor V. Queipo,et al.  Toward an optimal ensemble of kernel-based approximations with engineering applications , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[23]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[24]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[25]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[26]  Kevin Kok Wai Wong,et al.  Classification of adaptive memetic algorithms: a comparative study , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  James Smith,et al.  A tutorial for competent memetic algorithms: model, taxonomy, and design issues , 2005, IEEE Transactions on Evolutionary Computation.

[29]  Xin Yao,et al.  A framework for finding robust optimal solutions over time , 2013, Memetic Comput..

[30]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[31]  Kok Wai Wong,et al.  Surrogate-Assisted Evolutionary Optimization Frameworks for High-Fidelity Engineering Design Problems , 2005 .

[32]  Yew-Soon Ong,et al.  Curse and Blessing of Uncertainty in Evolutionary Algorithm Using Approximation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[33]  Khaled Rasheed,et al.  A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms , 2010 .

[34]  Xin Yao,et al.  Meta-Heuristic Algorithms in Car Engine Design: A Literature Survey , 2015, IEEE Transactions on Evolutionary Computation.

[35]  Salvador A. Pintos,et al.  Toward an optimal ensemble of kernel-based approximations with engineering applications , 2006 .

[36]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[37]  X. Yao,et al.  Combining landscape approximation and local search in global optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[38]  Andy J. Keane,et al.  Aircraft wing design using GA-based multi-level strategies , 2000 .

[39]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[40]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[41]  Chi-Keong Goh,et al.  Computational Intelligence in Optimization , 2010 .

[42]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[43]  Carretera de Valencia,et al.  The finite element method in electromagnetics , 2000 .

[44]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[45]  Jürgen Branke,et al.  On Using Surrogates with Genetic Programming , 2015, Evolutionary Computation.

[46]  Bernhard Sendhoff,et al.  Evolutionary Optimization with Dynamic Fidelity Computational Models , 2008, ICIC.

[47]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[48]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[49]  Loo Hay Lee,et al.  Memetic Algorithm for Real-Time Combinatorial Stochastic Simulation Optimization Problems With Performance Analysis , 2013, IEEE Transactions on Cybernetics.

[50]  C. Hirsch,et al.  Numerical Computation of Internal and External Flows. By C. HIRSCH. Wiley. Vol. 1, Fundamentals of Numerical Discretization. 1988. 515 pp. £60. Vol. 2, Computational Methods for Inviscid and Viscous Flows. 1990, 691 pp. £65. , 1991, Journal of Fluid Mechanics.

[51]  William E. Hart,et al.  Editorial Introduction Special Issue on Memetic Algorithms , 2004, Evolutionary Computation.

[52]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[53]  O. C. Zienkiewicz,et al.  The Finite Element Method for Solid and Structural Mechanics , 2013 .

[54]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[55]  Yew-Soon Ong,et al.  Memetic Computation—Past, Present & Future [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[56]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[57]  Kyriakos C. Giannakoglou,et al.  Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence , 2002 .

[58]  Bernhard Sendhoff,et al.  A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation , 2007, GECCO '07.

[59]  Andreas Zell,et al.  Evolution strategies assisted by Gaussian processes with improved preselection criterion , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[60]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[61]  Thomas W. Sederberg,et al.  Free-form deformation of solid geometric models , 1986, SIGGRAPH.

[62]  Gregory Piatetsky-Shapiro,et al.  High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality , 2000 .

[63]  Bernhard Sendhoff,et al.  On Evolutionary Optimization with Approximate Fitness Functions , 2000, GECCO.

[64]  Jürgen Branke,et al.  Faster convergence by means of fitness estimation , 2005, Soft Comput..

[65]  Sabine Coquillart,et al.  Extended free-form deformation: a sculpturing tool for 3D geometric modeling , 1990, SIGGRAPH.

[66]  Bernhard Sendhoff,et al.  Representing the Change - Free Form Deformation for Evolutionary Design Optimization , 2008, Evolutionary Computation in Practice.

[67]  Kaspar Willam,et al.  Review of The Finite Element Method for Solid and Structural Mechanics, 6th Edition, by O. C. Zienkiewicz and R. L. Taylor , 2006 .

[68]  Fred H. Lesh,et al.  Multi-dimensional least-squares polynomial curve fitting , 1959, CACM.

[69]  Dirk Thierens,et al.  An Adaptive Pursuit Strategy for Allocating Operator Probabilities , 2005, BNAIC.

[70]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[71]  Lam Thu Bui,et al.  Success in Evolutionary Computation , 2008 .

[72]  Yoel Tenne,et al.  A Versatile Surrogate-Assisted Memetic Algorithm for Optimization of Computationally Expensive Functions and its Engineering Applications , 2008 .

[73]  Yew-Soon Ong,et al.  Discovering Unique, Low-Energy Transition States Using Evolutionary Molecular Memetic Computing , 2013, IEEE Computational Intelligence Magazine.

[74]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[75]  David E. Goldberg,et al.  Probability matching, the magnitude of reinforcement, and classifier system bidding , 2004, Machine Learning.

[76]  Iain Murray,et al.  Introduction to Gaussian Processes , 2008 .

[77]  Christine A. Shoemaker,et al.  Local function approximation in evolutionary algorithms for the optimization of costly functions , 2004, IEEE Transactions on Evolutionary Computation.

[78]  Wenyin Gong,et al.  Engineering optimization by means of an improved constrained differential evolution , 2014 .

[79]  John Bell,et al.  A review of methods for the assessment of prediction errors in conservation presence/absence models , 1997, Environmental Conservation.

[80]  Sung-Bae Cho,et al.  An efficient genetic algorithm with less fitness evaluation by clustering , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).