Multiobjective Infill Criterion Driven Gaussian Process-Assisted Particle Swarm Optimization of High-Dimensional Expensive Problems

Model management plays an essential role in surrogate-assisted evolutionary optimization of expensive problems, since the strategy for selecting individuals for fitness evaluation using the real objective function has substantial influences on the final performance. Among many others, infill criterion driven Gaussian process (GP)-assisted evolutionary algorithms have been demonstrated competitive for optimization of problems with up to 50 decision variables. In this paper, a multiobjective infill criterion (MIC) that considers the approximated fitness and the approximation uncertainty as two objectives is proposed for a GP-assisted social learning particle swarm optimization algorithm. The MIC uses nondominated sorting for model management, thereby avoiding combining the approximated fitness and the approximation uncertainty into a scalar function, which is shown to be particularly important for high-dimensional problems, where the estimated uncertainty becomes less reliable. Empirical studies on 50-D and 100-D benchmark problems and a synthetic problem constructed from four real-world optimization problems demonstrate that the proposed MIC is more effective than existing scalar infill criteria for GP-assisted optimization given a limited computational budget.

[1]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[2]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[3]  Bernd Bischl,et al.  Model-Based Multi-objective Optimization: Taxonomy, Multi-Point Proposal, Toolbox and Benchmark , 2015, EMO.

[4]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[5]  Chi-Keong Goh,et al.  Multiproblem Surrogates: Transfer Evolutionary Multiobjective Optimization of Computationally Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[6]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[7]  Bo Liu,et al.  Global Optimization of Microwave Filters Based on a Surrogate Model-Assisted Evolutionary Algorithm , 2017, IEEE Transactions on Microwave Theory and Techniques.

[8]  Yew-Soon Ong,et al.  Evolutionary Optimization of Expensive Multiobjective Problems With Co-Sub-Pareto Front Gaussian Process Surrogates , 2019, IEEE Transactions on Cybernetics.

[9]  Thomas J. Santner,et al.  Two-Stage Sensitivity-Based Group Screening in Computer Experiments , 2012, Technometrics.

[10]  Xin Yao,et al.  Diversity Assessment in Many-Objective Optimization , 2017, IEEE Transactions on Cybernetics.

[11]  Meng-Sing Liou,et al.  Multiobjective optimization using coupled response surface model and evolutionary algorithm , 2004 .

[12]  Shigeru Obayashi,et al.  Expected Improvement of Penalty-Based Boundary Intersection for Expensive Multiobjective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[13]  Handing Wang Uncertainty in Surrogate Models , 2016, GECCO.

[14]  Ross D. Murch,et al.  A Compact Planar Printed MIMO Antenna Design , 2015, IEEE Transactions on Antennas and Propagation.

[15]  Ahmed Kattan,et al.  Surrogate Genetic Programming: A semantic aware evolutionary search , 2015, Inf. Sci..

[16]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[17]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[18]  Michèle Sebag,et al.  Comparison-Based Optimizers Need Comparison-Based Surrogates , 2010, PPSN.

[19]  Andy J. Keane,et al.  Evolutionary optimization for computationally expensive problems using Gaussian processes , 2001 .

[20]  Jie Tian,et al.  A self-adaptive similarity-based fitness approximation for evolutionary optimization , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[21]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[22]  M. Stein Statistical Interpolation of Spatial Data: Some Theory for Kriging , 1999 .

[23]  Bernhard Sendhoff,et al.  Individual-based Management of Meta-models for Evolutionary Optimization with Application to Three-Dimensional Blade Optimization , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[24]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[25]  Qingfu Zhang,et al.  Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model , 2010, IEEE Transactions on Evolutionary Computation.

[26]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[27]  Wolfgang Ponweiser,et al.  Clustered multiple generalized expected improvement: A novel infill sampling criterion for surrogate models , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[28]  Yuansheng Cheng,et al.  Expected Improvement Matrix-Based Infill Criteria for Expensive Multiobjective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[29]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[30]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[31]  Jürgen Branke,et al.  Efficient search for robust solutions by means of evolutionary algorithms and fitness approximation , 2006, IEEE Transactions on Evolutionary Computation.

[32]  Jürgen Branke,et al.  Faster convergence by means of fitness estimation , 2005, Soft Comput..

[33]  Handing Wang,et al.  Data-Driven Surrogate-Assisted Multiobjective Evolutionary Optimization of a Trauma System , 2016, IEEE Transactions on Evolutionary Computation.

[34]  J. Dennis,et al.  MANAGING APPROXIMATION MODELS IN OPTIMIZATION , 2007 .

[35]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[36]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[37]  Christopher K. I. Williams Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[38]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[39]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[40]  Alexandros Agapitos,et al.  GP made faster with semantic surrogate modelling , 2016, Inf. Sci..

[41]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[42]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[43]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[44]  KrauseAndreas,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008 .

[45]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[46]  Alain Ratle,et al.  Kriging as a surrogate fitness landscape in evolutionary optimization , 2001, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[47]  Ofer M. Shir,et al.  A Reduced-Cost SMS-EMOA Using Kriging, Self-Adaptation, and Parallelization , 2008, MCDM.

[48]  Xin Yao,et al.  Classification-assisted Differential Evolution for computationally expensive problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[49]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[50]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[51]  Nirupam Chakraborti,et al.  A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem , 2017 .

[52]  Bernhard Sendhoff,et al.  Comparing neural networks and Kriging for fitness approximation in evolutionary optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[53]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[54]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[55]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[56]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[57]  Tianyou Chai,et al.  Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems , 2019, IEEE Transactions on Cybernetics.

[58]  Jie Tian,et al.  Comparisons of different kernels in Kriging-assisted evolutionary expensive optimization , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[59]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[60]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[61]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[62]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[63]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[64]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[65]  Khaled Rasheed,et al.  A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms , 2010 .