Global Convergence of Radial Basis Function Trust Region Derivative-Free Algorithms

We analyze globally convergent derivative-free trust region algorithms relying on radial basis function interpolation models. Our results extend the recent work of Conn, Scheinberg, and Vicente [SIAM J. Optim., 20 (2009), pp. 387–415] to fully linear models that have a nonlinear term. We characterize the types of radial basis functions that fit in our analysis and thus show global convergence to first-order critical points for the ORBIT algorithm of Wild, Regis, and Shoemaker [SIAM J. Sci. Comput., 30 (2008), pp. 3197–3219]. Using ORBIT, we present numerical results for different types of radial basis functions on a series of test problems. We also demonstrate the use of ORBIT in finding local minima on a computationally expensive environmental engineering problem.

[1]  Paul P. Wang,et al.  MT3DMS: A Modular Three-Dimensional Multispecies Transport Model for Simulation of Advection, Dispersion, and Chemical Reactions of Contaminants in Groundwater Systems; Documentation and User's Guide , 1999 .

[2]  Katya Scheinberg,et al.  Geometry of interpolation sets in derivative free optimization , 2007, Math. Program..

[3]  R. Oeuvray.  Trust-region methods based on radial basis functions with application to biomedical imaging , 2005 .

[4]  Luís N. Vicente,et al.  Using Sampling and Simplex Derivatives in Pattern Search Methods , 2007, SIAM J. Optim..

[5]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[6]  Humberto Rocha,et al.  Incorporating minimum Frobenius norm models in direct search , 2010, Comput. Optim. Appl..

[7]  Christine A. Shoemaker,et al.  Derivative-free optimization algorithms for computationally expensive functions , 2009 .

[8]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[9]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[10]  M. Bierlaire,et al.  Boosters: A Derivative-Free Algorithm Based on Radial Basis Functions , 2009 .

[11]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[12]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[13]  Sandia Report,et al.  Revisiting Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2004 .

[14]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[15]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[16]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[17]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[18]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[19]  Stefan M. Wild.  MNH: A Derivative-Free Optimization Algorithm Using Minimal Norm Hessians , 2008 .

[20]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[21]  Jorge Nocedal,et al.  Wedge trust region methods for derivative free optimization , 2002, Math. Program..

[22]  Yan Zhang,et al.  Reducing Long‐Term Remedial Costs by Transport Modeling Optimization , 2006, Ground water.

[23]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[24]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[25]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[26]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[27]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[28]  T B Comstock,et al.  U. S. Geological Survey , 1907, Radiocarbon.

[29]  David G. Streets,et al.  BIOFUEL USE IN ASIA AND ACIDIFYING EMISSIONS1The above manuscript has been created by the University of Chicago as Operator of Argonne National Laboratory (“Argonne”) under Contract No. W-31-109-ENG-38 with the U.S. Department of Energy.1 , 1998 .

[30]  Mattias Björkman,et al.  Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions , 2000 .

[31]  Jorge Nocedal,et al.  On the geometry phase in model-based algorithms for derivative-free optimization , 2009, Optim. Methods Softw..

[32]  Katya Scheinberg,et al.  Recent progress in unconstrained nonlinear optimization without derivatives , 1997, Math. Program..

[33]  Tamara G. Kolda,et al.  Asynchronous parallel pattern search for nonlinear optimization , 2000 .

[34]  Richard C. Peralta,et al.  Final Cost and Performance Report Application of Flow and Transport Optimization Codes to Groundwater Pump and Treat Systems , 2004 .