Bayesian Approaches to Surrogate-Assisted Evolutionary Multi-objective Optimization: A Comparative Study

Both evolutionary Bayesian optimization (EBO) and Bayesian evolutionary optimization (BEO) are used to solve time-consuming optimization problems, where in the EBO method, the evolutionary algorithm is adopted to optimize the acquisition function for finding its optimal solution to be evaluated using the time consuming objective functions, while in the BEO approach, surrogates are utilized to replace the expensive objective functions and the acquisition function is used in the model management to choose individuals for evaluations using the expensive objective functions. In this paper, we perform comparisons on these two approaches to see which method is better for solving computationally expensive multi-objective problems. The expected improvement (EI) is used as the acquisition function in both EBO and BEO approaches. Two strategies are proposed to select individuals to be evaluated using the real computationally expensive objectives. The experimental results conducted on ZDT and DTLZ test problems showed that the BEO approach is more efficient than the EBO approach for solving multi-objective problems.

[1]  Bernhard Sendhoff,et al.  A Multiobjective Evolutionary Algorithm Using Gaussian Process-Based Inverse Modeling , 2015, IEEE Transactions on Evolutionary Computation.

[2]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[3]  Qingfu Zhang,et al.  Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model , 2010, IEEE Transactions on Evolutionary Computation.

[4]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[5]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[6]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[7]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[8]  Ying Tan,et al.  Multiobjective Infill Criterion Driven Gaussian Process-Assisted Particle Swarm Optimization of High-Dimensional Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[9]  Meng-Sing Liou,et al.  Multiobjective optimization using coupled response surface model and evolutionary algorithm , 2004 .

[10]  Ernesto Benini,et al.  A surrogate-assisted evolutionary algorithm based on the genetic diversity objective , 2015, Appl. Soft Comput..

[11]  Kalyanmoy Deb,et al.  A combined genetic adaptive search (GeneAS) for engineering design , 1996 .

[12]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[13]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[14]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[15]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[16]  Kaisa Miettinen,et al.  On Constraint Handling in Surrogate-Assisted Evolutionary Many-Objective Optimization , 2016, PPSN.

[17]  Ye Tian,et al.  A Knee Point-Driven Evolutionary Algorithm for Many-Objective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[18]  M. Herrera,et al.  Metamodel-assisted optimization based on multiple kernel regression for mixed variables , 2014, Structural and Multidisciplinary Optimization.

[19]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[20]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[21]  G. J. Mitchell,et al.  Principles and procedures of statistics: A biometrical approach , 1981 .

[22]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[23]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[24]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[25]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[26]  J. Dennis,et al.  MANAGING APPROXIMATION MODELS IN OPTIMIZATION , 2007 .

[27]  M. Farina.  A neural network based generalized response surface multiobjective evolutionary algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[28]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[29]  Ying Tan,et al.  Surrogate-assisted hierarchical particle swarm optimization , 2018, Inf. Sci..

[30]  Bo Liu,et al.  Global Optimization of Microwave Filters Based on a Surrogate Model-Assisted Evolutionary Algorithm , 2017, IEEE Transactions on Microwave Theory and Techniques.

[31]  Bernhard Sendhoff,et al.  A Reference Vector Guided Evolutionary Algorithm for Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[32]  D. Krige.  A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[33]  Jürgen Branke,et al.  Efficient search for robust solutions by means of evolutionary algorithms and fitness approximation , 2006, IEEE Transactions on Evolutionary Computation.

[34]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[35]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[36]  Ye Tian,et al.  A Classification-Based Surrogate-Assisted Evolutionary Algorithm for Expensive Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[37]  Xin Yao,et al.  Two_Arch2: An Improved Two-Archive Algorithm for Many-Objective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[38]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[39]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[40]  Shengxiang Yang,et al.  Pareto or Non-Pareto: Bi-Criterion Evolution in Multiobjective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[41]  Yuren Zhou,et al.  A Scalar Projection and Angle-Based Evolutionary Algorithm for Many-Objective Optimization Problems , 2019, IEEE Transactions on Cybernetics.

[42]  Marco Laumanns,et al.  Scalable multi-objective optimization test problems , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[43]  R. Haftka,et al.  Importing Uncertainty Estimates from One Surrogate to Another , 2009 .