Parallel Stochastic Global Optimization Using Radial Basis Functions

We develop a parallel implementation of a stochastic radial basis function (RBF) algorithm for global optimization by Regis and Shoemaker [Regis, R. G., C. A. Shoemaker. 2007a. A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput.19(4) 497--509]. The proposed parallel algorithm is suitable for the global optimization of computationally expensive objective functions and does not require derivatives. Each iteration of the algorithm consists of building an RBF model to approximate the expensive function and using this model to select multiple points for simultaneous function evaluation on multiple processors. The function evaluation points are selected from a set of random candidate points according to two criteria: estimated function value based on the RBF model, and minimum distance from previously evaluated points and previously selected points within each iteration. We compare the performance of our parallel stochastic RBF algorithm against alternative parallel global optimization methods, including two multistart parallel finite-difference quasi-Newton methods, a multistart implementation of Asynchronous Parallel Pattern Search [Hough, P., T. G. Kolda, V. J. Torczon. 2001. Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput.23(1) 134--156], a parallel implementation of Probabilistic Global Search Lausanne [Raphael, B., I. F. C. Smith. 2003. A direct stochastic algorithm for global search. Appl. Math. Comput.146 729--758], a parallel evolutionary algorithm, and a deterministic parallel RBF algorithm by Regis and Shoemaker [Regis, R. G., C. A. Shoemaker. 2007c. Parallel radial basis function methods for the global optimization of expensive functions. Eur. J. Oper. Res.182(2) 514--535]. We report good results for our parallel stochastic RBF method when using one, four, or eight processors in comparison with the alternatives on 20 test problems and on 3 optimization problems involving groundwater bioremediation.

[1]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[2]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[3]  Christine A. Shoemaker,et al.  Comparison of Optimization Methods for Ground-Water Bioremediation , 1999 .

[4]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[5]  Richard S. Barr,et al.  Feature Article - Reporting Computational Experiments with Parallel Algorithms: Issues, Measures, and Experts' Opinions , 1993, INFORMS J. Comput..

[6]  Kenny Q. Ye,et al.  Algorithmic construction of optimal symmetric Latin hypercube designs , 2000 .

[7]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[8]  Ian F. C. Smith,et al.  A direct stochastic algorithm for global search , 2003, Appl. Math. Comput..

[9]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[10]  Farrokh Mistree,et al.  Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization , 2001 .

[11]  Tamara G. Kolda,et al.  On the Convergence of Asynchronous Parallel Pattern Search , 2002, SIAM J. Optim..

[12]  Fred W. Glover,et al.  A Template for Scatter Search and Path Relinking , 1997, Artificial Evolution.

[13]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[14]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[15]  Tamara G. Kolda,et al.  Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2001, SIAM J. Sci. Comput..

[16]  Rafael Martí,et al.  Scatter Search: Diseño Básico y Estrategias avanzadas , 2002, Inteligencia Artif..

[17]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[18]  T Watson Layne,et al.  Multidisciplinary Optimization of a Supersonic Transport Using Design of Experiments Theory and Response Surface Modeling , 1997 .

[19]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[20]  Katya Scheinberg,et al.  Recent progress in unconstrained nonlinear optimization without derivatives , 1997, Math. Program..

[21]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[22]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[23]  M. J. D. Powell,et al.  On trust region methods for unconstrained minimization without derivatives , 2003, Math. Program..

[24]  Fred W. Glover,et al.  Scatter Search and Local Nlp Solvers: A Multistart Framework for Global Optimization , 2006, INFORMS J. Comput..

[25]  Christine A. Shoemaker,et al.  Improved Strategies for Radial basis Function Methods for Global Optimization , 2007, J. Glob. Optim..

[26]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[27]  Christine A. Shoemaker,et al.  Parallel radial basis function methods for the global optimization of expensive functions , 2007, Eur. J. Oper. Res..

[28]  Fabio Schoen,et al.  A wide class of test functions for global optimization , 1993, J. Glob. Optim..

[29]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[30]  Noel A. C. Cressie,et al.  Statistics for Spatial Data: Cressie/Statistics , 1993 .

[31]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[32]  Martin D. Buhmann,et al.  Radial Basis Functions , 2021, Encyclopedia of Mathematical Geosciences.

[33]  Christine A. Shoemaker,et al.  Comparison of function approximation, heuristic, and derivative‐based methods for automatic calibration of computationally expensive groundwater bioremediation models , 2005 .

[34]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[35]  Christine A. Shoemaker,et al.  Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions , 2005, J. Glob. Optim..