Efficient Bayesian Optimization Based on Parallel Sequential Random Embeddings

Bayesian optimization, which offers efficient parameter search, suffers from high computation cost if the parameters have high dimensionality because the search space expands and more trials are needed. One existing solution is an embedding method that enables the search to be restricted to a low-dimensional subspace, but this method works well only when the number of embedding dimensions closely match the that of effective dimensions, which affects the function value. However, in practical situations, the number of effective dimensions is unknown, and embedding into a low dimensional subspace to save computation cost often results in a search in a lower dimensional subspace than the effective dimensions. This study proposes a Bayesian optimization method that uses random embedding to remain efficient even if the embedded dimension is lower than the effective dimensions. By conducting parallel search in an initially low dimensional space and performing multiple cycles in which the search space is incrementally improved, the optimum solution can be efficiently found. An experiment on benchmark problems shows the effectiveness of the proposed method.