Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels

This paper presents and analyzes in detail an efficient search method based on evolutionary algorithms (EA) assisted by local Gaussian random field metamodels (GRFM). It is created for the use in optimization problems with one (or many) computationally expensive evaluation function(s). The role of GRFM is to predict objective function values for new candidate solutions by exploiting information recorded during previous evaluations. Moreover, GRFM are able to provide estimates of the confidence of their predictions. Predictions and their confidence intervals predicted by GRFM are used by the metamodel assisted EA. It selects the promising members in each generation and carries out exact, costly evaluations only for them. The extensive use of the uncertainty information of predictions for screening the candidate solutions makes it possible to significantly reduce the computational cost of singleand multiobjective EA. This is adequately demonstrated in this paper by means of mathematical test cases and a multipoint airfoil design in aerodynamics

[1]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[2]  Joshua D. Knowles,et al.  Bounded archiving using the lebesgue measure , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[3]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[4]  M. Pagel.  Encyclopedia of evolution , 2002 .

[5]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[6]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[7]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[8]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[9]  J. Dennis,et al.  MANAGING APPROXIMATION MODELS IN OPTIMIZATION , 2007 .

[10]  D. Dennis,et al.  A statistical method for global optimization , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[11]  Erich Novak,et al.  Global Optimization Using Hyperbolic Cross Points , 1996 .

[12]  Marios K. Karakasis,et al.  ON THE USE OF SURROGATE EVALUATION MODELS IN MULTI-OBJECTIVE EVOLUTIONARY ALGORITHMS , 2004 .

[13]  Michael Emmerich.  A rigorous analysis of two bi-criteria problem families with scalable curvature of the pareto fronts , 2005 .

[14]  Calyampudi R. Rao.  Handbook of statistics , 1980 .

[15]  Carlos M. Fonseca,et al.  Multiobjective genetic algorithms with application to control engineering problems. , 1995 .

[16]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[17]  William H. Press,et al.  Book-Review - Numerical Recipes in Pascal - the Art of Scientific Computing , 1989 .

[18]  Kyriakos C. Giannakoglou.  Designing Turbomachinery Blades Using Evolutionary Methods , 1999 .

[19]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[20]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[21]  Natalia Alexandrov,et al.  Multidisciplinary design optimization : state of the art , 1997 .

[22]  H. Schwefel.  Deep insight from simple models of evolution. , 2002, Bio Systems.

[23]  Marios K. Karakasis,et al.  Low-cost genetic optimization based on inexact pre-evaluations and the sensitivity analysis of design parameters , 2001 .

[24]  Nicola Beume,et al.  An EMO Algorithm Using the Hypervolume Measure as Selection Criterion , 2005, EMO.

[25]  Thomas Bartz-Beielstein,et al.  Validation and Optimization of an Elevator Simulation Model with Modern Search Heuristics , 2005 .

[26]  V. Bowman.  On the Relationship of the Tchebycheff Norm and the Efficient Frontier of Multiple-Criteria Objectives , 1976 .

[27]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[28]  Tomoyuki Hiroyasu,et al.  SPEA2+: Improving the Performance of the Strength Pareto Evolutionary Algorithm 2 , 2004, PPSN.

[29]  Günter Rudolph,et al.  A partial order approach to noisy fitness functions , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[30]  Kalyanmoy Deb,et al.  Computationally effective search and optimization procedure using coarse to fine approximations , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[31]  Hans-Paul Schwefel,et al.  Where Elitists Start Limping Evolution Strategies at Ridge Functions , 1998, PPSN.

[32]  Alain Ratle,et al.  Accelerating the Convergence of Evolutionary Algorithms by Fitness Landscape Approximation , 1998, PPSN.

[33]  Donald R. Jones,et al.  Global versus local search in constrained optimization of computer models , 1998 .

[34]  Nikolaus Hansen,et al.  Step-Size Adaption Based on Non-Local Use of Selection Information , 1994, PPSN.

[35]  Michael T. M. Emmerich,et al.  Design of Graph-Based Evolutionary Algorithms: A Case Study for Chemical Process Networks , 2001, Evolutionary Computation.

[36]  Thomas Bäck,et al.  Evaluating Multi-criteria Evolutionary Algorithms for Airfoil Optimisation , 2002, PPSN.

[37]  Nicola Beume,et al.  Multi-objective optimisation using S-metric selection: application to three-dimensional solution spaces , 2005, 2005 IEEE Congress on Evolutionary Computation.

[38]  Virginia Torczon,et al.  Numerical Optimization Using Computer Experiments , 1997 .

[39]  Peter Buchholz,et al.  Enhancing evolutionary algorithms with statistical selection procedures for simulation optimization , 2005, Proceedings of the Winter Simulation Conference, 2005..

[40]  T. Dobzhansky.  Genetics and the Origin of Species , 1937 .

[41]  J. Jakumeit,et al.  Parameter optimization of the sheet metal forming process using an iterative parallel Kriging algorithm , 2005 .

[42]  R. Adler,et al.  The Geometry of Random Fields , 1982 .

[43]  C. Zuppa.  Error estimates for modified local Shepard's interpolation formula , 2004 .

[44]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[45]  Mike Preuss,et al.  On the Extinction of Evolutionary Algorithm Subpopulations on Multimodal Landscapes , 2004, Informatica.

[46]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[47]  Yaochu Jin,et al.  Managing approximate models in evolutionary aerodynamic design optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[48]  Dr. Zbigniew Michalewicz,et al.  How to Solve It: Modern Heuristics , 2004 .

[49]  Hans-Georg Beyer,et al.  A Comparison of Evolution Strategies with Other Direct Search Methods in the Presence of Noise , 2003, Comput. Optim. Appl..

[50]  Andreas Zell,et al.  Evolution strategies assisted by Gaussian processes with improved preselection criterion , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[51]  D. Dennis,et al.  SDO : A Statistical Method for Global Optimization , 1997 .

[52]  Kyriakos C. Giannakoglou,et al.  Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence , 2002 .

[53]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[54]  D. G. Krige,et al.  A study of gold and uranium distribution patterns in the Klerksdorp gold field , 1966 .

[55]  Raphael T. Haftka,et al.  Recent Advances in Approximation Concepts for Optimum Structural Design , 1993 .

[56]  Thomas Bäck,et al.  Industrial Applications of Evolutionary Algorithms: A Comparison to Traditional Methods , 2002 .

[57]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[58]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[59]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[60]  A. Charnes,et al.  Goal programming and multiple objective optimizations: Part 1 , 1977 .

[61]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[62]  Stephen G. Nash,et al.  SUMT (Revisited) , 1998, Oper. Res..

[63]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[64]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[65]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[66]  Michael Emmerich,et al.  Metamodel Assisted Multiobjective Optimisation Strategies and their Application in Airfoil Design , 2004 .

[67]  Mike Preuss,et al.  Counteracting genetic drift and disruptive recombination in (μpluskommaλ)-EA on multimodal fitness landscapes , 2005, GECCO '05.

[68]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[69]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[70]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[71]  W. D. Evans,et al.  PARTIAL DIFFERENTIAL EQUATIONS , 1941 .

[72]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[73]  David J. C. Mackay,et al.  Introduction to Monte Carlo Methods , 1998, Learning in Graphical Models.

[74]  Barbara Haas Margolius,et al.  Permutations with Inversions , 2001 .

[75]  D. Myers.  Kriging, cokriging, radial basis functions and the role of positive definiteness , 1992 .

[76]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[77]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[78]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[79]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[80]  Ran Raz.  On the Complexity of Matrix Product , 2003, SIAM J. Comput..

[81]  Zelda B. Zabinsky,et al.  Stochastic Methods for Practical Global Optimization , 1998, J. Glob. Optim..

[82]  Frank Kursawe,et al.  Grundlegende empirische Untersuchungen der Parameter von Evolutionsstrategien - Metastrategien , 1999 .

[83]  V. Torczon,et al.  Direct search methods: then and now , 2000 .

[84]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[85]  P. Papalambros,et al.  A NOTE ON WEIGHTED CRITERIA METHODS FOR COMPROMISE SOLUTIONS IN MULTI-OBJECTIVE OPTIMIZATION , 1996 .

[86]  Philip E. Gill,et al.  Practical optimization , 1981 .

[87]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[88]  Claude-Pierre Jeannerod,et al.  On the complexity of polynomial matrix computations , 2003, ISSAC '03.

[89]  Mark Fleischer,et al.  The measure of pareto optima: Applications to multi-objective metaheuristics , 2003 .

[90]  Iain Murray.  Introduction To Gaussian Processes , 2008 .

[91]  Jan Paredis,et al.  Coevolutionary computation , 1995 .

[92]  Eckart Zitzler,et al.  Evolutionary algorithms for multiobjective optimization: methods and applications , 1999 .

[93]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[94]  Frank Hoffmeister,et al.  Problem-Independent Handling of Constraints by Use of Metric Penalty Functions , 1996, Evolutionary Programming.

[95]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[96]  Marc Schoenauer,et al.  Constrained GA Optimization , 1993, ICGA.

[97]  M. Fleischer,et al.  The Measure of Pareto Optima , 2003, EMO.

[98]  David W. Corne,et al.  Properties of an adaptive archiving algorithm for storing nondominated vectors , 2003, IEEE Trans. Evol. Comput..

[99]  Andy J. Keane,et al.  Metamodeling Techniques For Evolutionary Optimization of Computationally Expensive Problems: Promises and Limitations , 1999, GECCO.

[100]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[101]  Christine A. Shoemaker,et al.  Local function approximation in evolutionary algorithms for the optimization of costly functions , 2004, IEEE Transactions on Evolutionary Computation.

[102]  Liang Shi,et al.  Multiobjective GA optimization using reduced models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[103]  Michael Emmerich,et al.  TEA - A C++ Library for the Design of Evolutionary Algorithms , 2001 .

[104]  Christopher M. Siefert,et al.  Model-Assisted Pattern Search , 2000 .

[105]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[106]  D. Ackley.  A connectionist machine for genetic hillclimbing , 1987 .

[107]  Aiaa Oo.  Institute for Computer Applications in Science , 2000 .

[108]  Bernhard Sendhoff,et al.  Structure optimization of neural networks for evolutionary design optimization , 2005, Soft Comput..

[109]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[110]  Jürgen Branke,et al.  Faster convergence by means of fitness estimation , 2005, Soft Comput..

[111]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[112]  A. Patera,et al.  ICASE Report No . 93-50 191510 IC S 2 O Years ofExcellence SURROGATES FOR NUMERICAL SIMULATIONS ; OPTIMIZATION OF EDDY-PROMOTER HEAT EXCHANGERS , 1993 .