Cooling Strategies for the Moment-Generating Function in Bayesian Global Optimization

Bayesian Global Optimization algorithm is designed to optimize expensive objective functions with small evaluation budget. This algorithm employs a surrogate model and assesses the potential improvement of unseen solutions through the so-called infill-criterion. A novel infill-criterion proposed in our previous work is derived from the moment-generating function of the improvement. In contrast to other techniques, it features a continuous parameter that can be used to adjust the exploration-exploitation tradeoff smoothly. In this work, two cooling strategies (linear and exponential) are adopted to enhance the explorative behavior in the early stage of the search and the exploitative effect in the final converging stage. Moreover, the initial temperature and cooling speed are investigated on some selected multi-modal functions, showing that the good setting of those two parameters depends on the problems specifics. The proposed Bayesian optimization with cooling strategy is tested on well-known BBOB benchmark. The results shows that without tuning the initial temperature and cooling speed, the proposed approach improves the performance on a range of multi-modal functions as compared to the commonly used expected improvement criterion.