Computing 3-D Expected Hypervolume Improvement and Related Integrals in Asymptotically Optimal Time

The Expected Hypervolume Improvement EHVI is a frequently used infill criterion in surrogate-assisted multi-criterion optimization. It needs to be frequently called during the execution of such algorithms. Despite recent advances in improving computational efficiency, its running time for three or more objectives has remained in $$On^d$$ for $$d\ge 3$$, where d is the number of objective functions and n is the size of the incumbent Pareto-front approximation. This paper proposes a new integration scheme, which makes it possible to compute the EHVI in $$\varTheta n \log n$$ optimal time for the important three-objective case $$d=3$$. The new scheme allows for a generalization to higher dimensions and for computing the Probability of Improvement PoI integral efficiently. It is shown, both theoretically and empirically, that the hidden constant in the asymptotic notation is small. Empirical speed comparisons were designed between the C++ implementations of the new algorithm which will be in the public domain and those recently published by competitors, on randomly-generated non-dominated fronts of size 10, 100, and 1000. The experiments include the analysis of batch computations, in which only the parameters of the probability distribution change but the incumbent Pareto-front approximation stays the same. Experimental results show that the new algorithm is always faster than the other algorithms, sometimes over $$10^4$$ times faster.

[1]  Shigeru Obayashi,et al.  Kriging model based many-objective optimization with efficient calculation of expected hypervolume improvement , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[2]  Ofer M. Shir,et al.  The application of evolutionary multi-criteria optimization to dynamic molecular alignment , 2007, 2007 IEEE Congress on Evolutionary Computation.

[3]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[4]  E. Vázquez,et al.  Convergence properties of the expected improvement algorithm with fixed mean and covariance functions , 2007, 0712.3744.

[5]  Michael T. M. Emmerich,et al.  Hypervolume-based expected improvement: Monotonicity properties and exact computation , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[6]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[7]  Thomas Bartz-Beielstein,et al.  A Case Study on Multi-Criteria Optimization of an Event Detection Software under Limited Budgets , 2013, EMO.

[8]  Anne Auger,et al.  Hypervolume-based multiobjective optimization: Theoretical foundations and practical implications , 2012, Theor. Comput. Sci..

[9]  Thomas J. Santner,et al.  Multiobjective optimization of expensive-to-evaluate deterministic computer simulator models , 2016, Comput. Stat. Data Anal..

[10]  Thomas Bäck,et al.  Preference-based multiobjective optimization using truncated expected hypervolume improvement , 2016, 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[11]  Wolfgang Ponweiser,et al.  On Expected-Improvement Criteria for Model-based Multi-objective Optimization , 2010, PPSN.

[12]  Carlos M. Fonseca,et al.  A Box Decomposition Algorithm to Compute the Hypervolume Indicator , 2015, Comput. Oper. Res..

[13]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[14]  Shigeru Obayashi,et al.  Kriging-surrogate-based optimization considering expected hypervolume improvement in non-constrained many-objective test problems , 2013, 2013 IEEE Congress on Evolutionary Computation.

[15]  Thomas Bäck,et al.  Truncated expected hypervolume improvement: Exact computation and application , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[16]  Thomas Bäck,et al.  Efficient multi-criteria optimization on noisy machine learning problems , 2015, Appl. Soft Comput..

[17]  Joshua D. Knowles,et al.  Multiobjective Optimization on a Budget of 250 Evaluations , 2005, EMO.

[18]  Shigeru Obayashi,et al.  Comparison of the criteria for updating Kriging response surface models in multi-objective optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[19]  Tom Dhaene,et al.  Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization , 2014, J. Glob. Optim..

[20]  Thomas Bäck,et al.  Expected hypervolume improvement algorithm for PID controller tuning and the multiobjective dynamical control of a biogas plant , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[21]  Andy J. Keane,et al.  Statistical Improvement Criteria for Use in Multiobjective Design Optimization , 2006 .

[22]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[23]  Hao Wang,et al.  A Multicriteria Generalization of Bayesian Global Optimization , 2016, Advances in Stochastic and Deterministic Global Optimization.

[24]  Carlos M. Fonseca,et al.  Computing Hypervolume Contributions in Low Dimensions: Asymptotically Optimal Algorithm and Complexity Results , 2011, EMO.

[25]  Michael T. M. Emmerich,et al.  Faster Exact Algorithms for Computing Expected Hypervolume Improvement , 2015, EMO.

[26]  Shigeru Obayashi,et al.  Updating Kriging Surrogate Models Based on the Hypervolume Indicator in Multi-Objective Optimization , 2013 .