A multi-point mechanism of expected hypervolume improvement for parallel multi-objective bayesian global optimization

The technique of parallelization is a trend in the field of Bayesian global optimization (BGO) and is important for real-world applications because it can make full use of CPUs and speed up the execution times. This paper proposes a multi-point mechanism of the expected hypervolume improvement (EHVI) for multi-objective BGO (MOBGO) by the utilization of the truncated EHVI (TEHVI). The basic idea is to divide the objective space into several sub-objective spaces and then search for the optimal solutions in each sub-objective space by using the TEHVI as the infill criterion. We studied the performance of the proposed algorithm and performed comparisons with Kriging believer technique (KB) on five scientific benchmarks and a real-world application problem (i.e., a low-fidelity multi-objective airfoil optimization design). The stochastic experimental results show that the proposed algorithm performs better than the KB with respect to the hypervolume indicator, indicating that the proposed method provides an efficient parallelization technique for MOBGO.

[1]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[2]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[3]  William J. Welch,et al.  Computer experiments and global optimization , 1997 .

[4]  Thomas Bäck,et al.  Multi-Objective Bayesian Global Optimization using expected hypervolume improvement gradient , 2019, Swarm Evol. Comput..

[5]  Qingfu Zhang,et al.  Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model , 2010, IEEE Transactions on Evolutionary Computation.

[6]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[7]  H. Sobieczky.  Parametric Airfoils and Wings , 1999 .

[8]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[9]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[10]  Neil D. Lawrence,et al.  Batch Bayesian Optimization via Local Penalization , 2015, AISTATS.

[11]  Carlos M. Fonseca,et al.  Computing 3-D Expected Hypervolume Improvement and Related Integrals in Asymptotically Optimal Time , 2017, EMO.

[12]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[13]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[14]  Pramudita Satria Palar,et al.  Benchmarking Multi-Objective Bayesian Global Optimization Strategies for Aerodynamic Design , 2018 .

[15]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[16]  Wolfgang Ponweiser,et al.  On Expected-Improvement Criteria for Model-based Multi-objective Optimization , 2010, PPSN.

[17]  David Ginsbourger,et al.  Efficient batch-sequential Bayesian optimization with moments of truncated Gaussian vectors , 2016, 1609.02700.

[18]  M. Emmerich,et al.  The computation of the expected improvement in dominated hypervolume of Pareto front approximations , 2008 .

[19]  Bruce E. Stuckman,et al.  A global search method for optimizing nonlinear systems , 1988, IEEE Trans. Syst. Man Cybern..

[20]  Peter I. Frazier,et al.  Parallel Bayesian Global Optimization of Expensive Functions , 2016, Oper. Res..

[21]  D. Ginsbourger,et al.  Kriging is well-suited to parallelize optimization , 2010 .

[22]  Thomas Bäck,et al.  Preference-based multiobjective optimization using truncated expected hypervolume improvement , 2016, 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[23]  David Gaudrie,et al.  Targeting solutions in Bayesian multi-objective optimization: sequential and batch versions , 2018, Annals of Mathematics and Artificial Intelligence.

[24]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[25]  Thomas Bäck,et al.  Multi-objective aerodynamic design with user preference using truncated expected hypervolume improvement , 2018, GECCO.

[26]  Michael T. M. Emmerich,et al.  Faster Exact Algorithms for Computing Expected Hypervolume Improvement , 2015, EMO.

[27]  Thomas Bäck,et al.  Truncated expected hypervolume improvement: Exact computation and application , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[28]  Nicola Beume,et al.  An EMO Algorithm Using the Hypervolume Measure as Selection Criterion , 2005, EMO.

[29]  Shigeru Obayashi,et al.  Comparison of the criteria for updating Kriging response surface models in multi-objective optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[30]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[31]  Michael T. M. Emmerich,et al.  Test Problems Based on Lamé Superspheres , 2007, EMO.

[32]  Shigeru Obayashi,et al.  Expected Improvement of Penalty-Based Boundary Intersection for Expensive Multiobjective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[33]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.