GP-DEMO: Differential Evolution for Multiobjective Optimization based on Gaussian Process models

This paper proposes a novel surrogate-model-based multiobjective evolutionary algorithm called Differential Evolution for Multiobjective Optimization based on Gaussian Process models (GP-DEMO). The algorithm is based on the newly defined relations for comparing solutions under uncertainty. These relations minimize the possibility of wrongly performed comparisons of solutions due to inaccurate surrogate model approximations. The GP-DEMO algorithm was tested on several benchmark problems and two computationally expensive real-world problems. To be able to assess the results we compared them with another surrogate-model-based algorithm called Generational Evolution Control (GEC) and with the Differential Evolution for Multiobjective Optimization (DEMO). The quality of the results obtained with GP-DEMO was similar to the results obtained with DEMO, but with significantly fewer exactly evaluated solutions during the optimization process. The quality of the results obtained with GEC was lower compared to the quality gained with GP-DEMO and DEMO, mainly due to wrongly performed comparisons of the inaccurately approximated solutions.

[1]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[2]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[3]  J Kocijan,et al.  Application of Gaussian processes for black-box modelling of biosystems. , 2007, ISA transactions.

[4]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[5]  R. Lyndon While,et al.  A Scalable Multi-objective Test Problem Toolkit , 2005, EMO.

[6]  A. Žilinskas,et al.  On the use of statistical models of multimodal functions for the construction of the optimization algorithms , 1980 .

[7]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[8]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[9]  D. Krige.  A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[10]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[11]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[12]  Jack P. C. Kleijnen,et al.  Application-driven sequential designs for simulation experiments: Kriging metamodelling , 2004, J. Oper. Res. Soc..

[13]  Bogdan Filipic,et al.  DEMO: Differential Evolution for Multiobjective Optimization , 2005, EMO.

[14]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[15]  David W. Coit,et al.  Practical solutions for multi-objective optimization: An application to system reliability design problems , 2007, Reliab. Eng. Syst. Saf..

[16]  Liang Shi,et al.  Multiobjective GA optimization using reduced models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[17]  Darko Vrečko,et al.  Multi-criteria analyses of wastewater treatment bio-processes under an uncertainty and a multiplicity of steady states. , 2012, Water research.

[18]  Wolfgang Ponweiser,et al.  On Expected-Improvement Criteria for Model-based Multi-objective Optimization , 2010, PPSN.

[19]  Roman Neruda,et al.  An Evolutionary Strategy for Surrogate-Based Multiobjective Optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[20]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[21]  R. Gunst.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[22]  Michael Emmerich,et al.  Metamodel Assisted Multiobjective Optimisation Strategies and their Application in Airfoil Design , 2004 .

[23]  A. O'Hagan,et al.  Curve Fitting and Optimal Design for Prediction , 1978 .

[24]  Heike Trautmann,et al.  New Uncertainty Handling Strategies in Multi-objective Evolutionary Optimization , 2010, PPSN.

[25]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[26]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[27]  Layne T. Watson,et al.  Efficient global optimization algorithm assisted by multiple surrogate techniques , 2012, Journal of Global Optimization.

[28]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[29]  J. Rodgers,et al.  Thirteen ways to look at the correlation coefficient , 1988 .

[30]  Bojan Likar,et al.  Predictive control of a gas-liquid separation plant based on a Gaussian process model , 2007, Comput. Chem. Eng..

[31]  Roman Neruda,et al.  ASM-MOMA: Multiobjective memetic algorithm with aggregate surrogate model , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[32]  R. Vertnik,et al.  Simulation of continuous casting of steel by a meshless technique , 2009 .

[33]  Rainer Storn,et al.  Differential Evolution-A simple evolution strategy for fast optimization , 1997 .

[34]  G. Matheron.  The intrinsic random functions and their applications , 1973, Advances in Applied Probability.

[35]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[36]  Yaochu Jin,et al.  Managing approximate models in evolutionary aerodynamic design optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[37]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[38]  J. Weston,et al.  Approximation Methods for Gaussian Process Regression , 2007 .

[39]  R JonesDonald,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998 .

[40]  D. Mackay,et al.  Introduction to Gaussian processes , 1998 .

[41]  Andreas Zell,et al.  Evolution strategies assisted by Gaussian processes with improved preselection criterion , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[42]  D. W. Zimmerman.  Teacher’s Corner: A Note on Interpretation of the Paired-Samples t Test , 1997 .

[43]  Kalyanmoy Deb,et al.  An Evolutionary Multi-objective Adaptive Meta-modeling Procedure Using Artificial Neural Networks , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[44]  Günter Rudolph,et al.  A partial order approach to noisy fitness functions , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[45]  Philipp Limbourg,et al.  An optimization algorithm for imprecise multi-objective problem functions , 2005, 2005 IEEE Congress on Evolutionary Computation.

[46]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[47]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[48]  D. Grierson,et al.  Optimal sizing, geometrical and topological design using a genetic algorithm , 1993 .

[49]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[50]  B. Filipič,et al.  IN SEARCH FOR AN EFFICIENT PARAMETER TUNING METHOD FOR STEEL CASTING , 2004 .

[51]  Michael T. M. Emmerich,et al.  Hypervolume-based expected improvement: Monotonicity properties and exact computation , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[52]  Matjaz Depolli,et al.  Computer‐Simulated Alternative Modes of U‐Wave Genesis , 2007, Journal of cardiovascular electrophysiology.

[53]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[54]  R. L. Hardy.  Multiquadric equations of topography and other irregular surfaces , 1971 .

[55]  Arthur C. Sanderson,et al.  DE-AEC: A differential evolution algorithm based on adaptive evolution control , 2007, 2007 IEEE Congress on Evolutionary Computation.

[56]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[57]  Philipp Limbourg,et al.  Multi-objective Optimization of Problems with Epistemic Uncertainty , 2005, EMO.

[58]  S. Azarm,et al.  Improving multi-objective genetic algorithms with adaptive design of experiments and online metamodeling , 2009 .

[59]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).