Faster Exact Algorithms for Computing Expected Hypervolume Improvement

This paper is about computing the expected improvement of the hypervolume indicator given a Pareto front approximation and a predictive multivariate Gaussian distribution of a new candidate point. It is frequently used as an infill or prescreening criterion in multiobjective optimization with expensive function evaluations where predictions are provided by Kriging or Gaussian process surrogate models. The expected hypervolume improvement has good properties as an infill criterion, but exact algorithms for its computation have so far been very time consuming even for the two and three objective case. This paper introduces faster exact algorithms for computing the expected hypervolume improvement for independent Gaussian distributions. A new general computation scheme is introduced and a lower bound for the time complexity. By providing new algorithms, upper bounds for the time complexity for problems with two as well as three objectives are improved. For the 2-D case the time complexity bound is reduced from previously \(O(n^3 \log n)\) to \(O(n^2)\). For the 3-D case the new upper bound of \(O(n^3)\) is established; previously \(O(n^4 \log n)\). It is also shown how an efficient implementation of these new algorithms can lead to a further reduction of running time. Moreover it is shown how to process batches of multiple predictive distributions efficiently. The theoretical analysis is complemented by empirical speed comparisons of C++ implementations of the new algorithms to existing implementations of other exact algorithms.

[1]  Robert Schaefer Parallel Problem Solving from Nature - PPSN XI, 11th International Conference, Kraków, Poland, September 11-15, 2010. Proceedings, Part II , 2010, PPSN.

[2]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[3]  Thomas Bartz-Beielstein,et al.  A Case Study on Multi-Criteria Optimization of an Event Detection Software under Limited Budgets , 2013, EMO.

[4]  Wolfgang Ponweiser,et al.  On Expected-Improvement Criteria for Model-based Multi-objective Optimization , 2010, PPSN.

[5]  Michael T. M. Emmerich,et al.  Hypervolume-based expected improvement: Monotonicity properties and exact computation , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[6]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[7]  Shigeru Obayashi,et al.  Comparison of the criteria for updating Kriging response surface models in multi-objective optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[8]  Michael T. M. Emmerich,et al.  Faster Computation of Expected Hypervolume Improvement , 2014, ArXiv.

[9]  Tom Dhaene,et al.  Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization , 2014, J. Glob. Optim..

[10]  Carlos M. Fonseca,et al.  Computing Hypervolume Contributions in Low Dimensions: Asymptotically Optimal Algorithm and Complexity Results , 2011, EMO.

[11]  Shigeru Obayashi,et al.  Kriging model based many-objective optimization with efficient calculation of expected hypervolume improvement , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[12]  Thomas Bäck,et al.  Parallel Problem Solving from Nature — PPSN V , 1998, Lecture Notes in Computer Science.

[13]  E. Vázquez,et al.  Convergence properties of the expected improvement algorithm with fixed mean and covariance functions , 2007, 0712.3744.

[14]  H. Zimmermann Towards global optimization 2: L.C.W. DIXON and G.P. SZEGÖ (eds.) North-Holland, Amsterdam, 1978, viii + 364 pages, US $ 44.50, Dfl. 100,-. , 1979 .

[15]  Shigeru Obayashi,et al.  Kriging-surrogate-based optimization considering expected hypervolume improvement in non-constrained many-objective test problems , 2013, 2013 IEEE Congress on Evolutionary Computation.

[16]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[17]  Ofer M. Shir,et al.  The application of evolutionary multi-criteria optimization to dynamic molecular alignment , 2007, 2007 IEEE Congress on Evolutionary Computation.

[18]  Thomas Bäck,et al.  Efficient multi-criteria optimization on noisy machine learning problems , 2015, Appl. Soft Comput..

[19]  Nicola Beume,et al.  On the Complexity of Computing the Hypervolume Indicator , 2009, IEEE Transactions on Evolutionary Computation.