SOP: parallel surrogate global optimization with Pareto center selection for computationally expensive single objective problems

This paper presents a parallel surrogate-based global optimization method for computationally expensive objective functions that is more effective for larger numbers of processors. To reach this goal, we integrated concepts from multi-objective optimization and tabu search into, single objective, surrogate optimization. Our proposed derivative-free algorithm, called SOP, uses non-dominated sorting of points for which the expensive function has been previously evaluated. The two objectives are the expensive function value of the point and the minimum distance of the point to previously evaluated points. Based on the results of non-dominated sorting, P points from the sorted fronts are selected as centers from which many candidate points are generated by random perturbations. Based on surrogate approximation, the best candidate point is subsequently selected for expensive evaluation for each of the P centers, with simultaneous computation on P processors. Centers that previously did not generate good solutions are tabu with a given tenure. We show almost sure convergence of this algorithm under some conditions. The performance of SOP is compared with two RBF based methods. The test results show that SOP is an efficient method that can reduce time required to find a good near optimal solution. In a number of cases the efficiency of SOP is so good that SOP with 8 processors found an accurate answer in less wall-clock time than the other algorithms did with 32 processors.

[1]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[2]  Christine A. Shoemaker,et al.  Watershed calibration using multistart local optimization and evolutionary optimization with radial basis function approximation , 2007 .

[3]  Tamara G. Kolda,et al.  Asynchronous parallel pattern search for nonlinear optimization , 2000 .

[4]  Haitao Liu,et al.  Global optimization of expensive black box functions using potential Lipschitz constants and response surfaces , 2015, J. Glob. Optim..

[5]  Julius Zilinskas,et al.  Globally-biased Disimpl algorithm for expensive global optimization , 2014, Journal of Global Optimization.

[6]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[7]  Bernd Bischl,et al.  MOI-MBO: Multiobjective Infill for Parallel Model-Based Optimization , 2014, LION.

[8]  N. Zheng,et al.  Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models , 2006, J. Glob. Optim..

[9]  Christine A. Shoemaker,et al.  Parallel Stochastic Global Optimization Using Radial Basis Functions , 2009, INFORMS J. Comput..

[10]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[11]  Christine A. Shoemaker,et al.  Local function approximation in evolutionary algorithms for the optimization of costly functions , 2004, IEEE Transactions on Evolutionary Computation.

[12]  Christine A. Shoemaker,et al.  Comparison of Optimization Methods for Ground-Water Bioremediation , 1999 .

[13]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[14]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[15]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[16]  Jasper A Vrugt,et al.  Improved evolutionary optimization from genetically adaptive multimethod search , 2007, Proceedings of the National Academy of Sciences.

[17]  Layne T. Watson,et al.  Efficient global optimization algorithm assisted by multiple surrogate techniques , 2012, Journal of Global Optimization.

[18]  Xin Chen,et al.  Multiple-optima search method based on a metamodel and mathematical morphology , 2016 .

[19]  Christine A. Shoemaker,et al.  Comparison of function approximation, heuristic, and derivative‐based methods for automatic calibration of computationally expensive groundwater bioremediation models , 2005 .

[20]  Kenny Q. Ye,et al.  Algorithmic construction of optimal symmetric Latin hypercube designs , 2000 .

[21]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[22]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[23]  Taimoor Akhtar,et al.  Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection , 2016, J. Glob. Optim..

[24]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[25]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[26]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[27]  Andy J. Keane,et al.  On the Design of Optimization Strategies Based on Global Response Surface Approximation Models , 2005, J. Glob. Optim..

[28]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[29]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .