Global Convergence of Radial Basis Function Trust-Region Algorithms for Derivative-Free Optimization

We analyze globally convergent, derivative-free trust-region algorithms relying on radial basis function interpolation models. Our results extend the recent work of Conn, Scheinberg, and Vicente [SIAM J. Optim., 20 (2009), pp. 387--415] to fully linear models that have a nonlinear term. We characterize the types of radial basis functions that fit in our analysis and thus show global convergence to first-order critical points for the ORBIT algorithm of Wild, Regis, and Shoemaker [SIAM J. Sci. Comput., 30 (2008), pp. 3197--3219]. Using ORBIT, we present numerical results for different types of radial basis functions on a series of test problems. We also demonstrate the use of ORBIT in finding local minima on a computationally expensive environmental engineering problem involving remediation of contaminated groundwater.

[1]  Jorge Nocedal,et al.  Wedge trust region methods for derivative free optimization , 2002, Math. Program..

[2]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[3]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[4]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[5]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[6]  C. T. Kelley,et al.  Implicit Filtering , 2011 .

[7]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[8]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[9]  Stefan M. Wild.  MNH: A Derivative-Free Optimization Algorithm Using Minimal Norm Hessians , 2008 .

[10]  Tamara G. Kolda,et al.  Asynchronous parallel pattern search for nonlinear optimization , 2000 .

[11]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[12]  Paul P. Wang,et al.  MT3DMS: A Modular Three-Dimensional Multispecies Transport Model for Simulation of Advection, Dispersion, and Chemical Reactions of Contaminants in Groundwater Systems; Documentation and User's Guide , 1999 .

[13]  M. Bierlaire,et al.  Boosters: A Derivative-Free Algorithm Based on Radial Basis Functions , 2009 .

[14]  R. Oeuvray.  Trust-region methods based on radial basis functions with application to biomedical imaging , 2005 .

[15]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[16]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[17]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[18]  Stefan M. Wild,et al.  Estimating Derivatives of Noisy Simulations , 2012, TOMS.

[19]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[20]  Luís N. Vicente,et al.  Using Sampling and Simplex Derivatives in Pattern Search Methods , 2007, SIAM J. Optim..

[21]  Katya Scheinberg,et al.  Geometry of interpolation sets in derivative free optimization , 2007, Math. Program..

[22]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[23]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[24]  Humberto Rocha,et al.  Incorporating minimum Frobenius norm models in direct search , 2010, Comput. Optim. Appl..

[25]  Christine A. Shoemaker,et al.  Derivative-free optimization algorithms for computationally expensive functions , 2009 .

[26]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[27]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[28]  Yan Zhang,et al.  Reducing Long‐Term Remedial Costs by Transport Modeling Optimization , 2006, Ground water.

[29]  Richard C. Peralta,et al.  Final Cost and Performance Report Application of Flow and Transport Optimization Codes to Groundwater Pump and Treat Systems , 2004 .

[30]  Sandia Report,et al.  Revisiting Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2004 .

[31]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[32]  Katya Scheinberg,et al.  Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization , 2012, Mathematical Programming.

[33]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[34]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[35]  Mattias Björkman,et al.  Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions , 2000 .

[36]  Jorge Nocedal,et al.  On the geometry phase in model-based algorithms for derivative-free optimization , 2009, Optim. Methods Softw..

[37]  Katya Scheinberg,et al.  Recent progress in unconstrained nonlinear optimization without derivatives , 1997, Math. Program..