Optimization combining derivative-free global exploration with derivative-based local refinement

This paper proposes a hybrid optimization scheme combining an efficient (and, under the appropriate assumptions, provably globally convergent) derivative-free optimization algorithm (dubbed A-DOGS), to globally explore expensive noncon-vex functions, with a new derivative-based local optimization algorithm, to maximally accelerate local convergence from promising feasible points discovered in parameter space. The resulting hybrid optimization scheme proceeds iteratively, automatically shifting between (derivative-free) global exploration and (derivative-based) local refinement as appropriate. The new derivative-based local refinement method implemented uses the Voronoi partitioning of all existing datapoints at each iteration to establish a “modified trust-region” around the current best point, within which derivative-based optimization is considered. The resulting algorithm is analyzed, and its global convergence is proven under certain assumptions on the objective function. Finally, the algorithm is applied to nonconvex optimization problems with multiple local minima, and its computational cost compared with that of the original A-DOGS algorithm.