A surrogate management framework using rigorous trust-region steps

Surrogate models are frequently used in the optimization engineering community as convenient approaches to deal with functions for which evaluations are expensive or noisy, or lack convexity. These methodologies do not typically guarantee any type of convergence under reasonable assumptions. In this article, we will show how to incorporate the use of surrogate models, heuristics, or any other process of attempting a function value decrease in trust-region algorithms for unconstrained derivative-free optimization, in a way that global convergence of the latter algorithms to stationary points is retained. Our approach follows the lines of search/poll direct-search methods and corresponding surrogate management frameworks, both in algorithmic design and in the form of organizing the convergence theory.

[1]  J. S ndergaard,et al.  Optimization Using Surrogate Models-by the Space Mapping Technique , 2003 .

[2]  Jorge Nocedal,et al.  On the geometry phase in model-based algorithms for derivative-free optimization , 2009, Optim. Methods Softw..

[3]  Andy J. Keane,et al.  Computational Approaches for Aerospace Design: The Pursuit of Excellence , 2005 .

[4]  John L. Nazareth,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[5]  Giuseppe Nicosia,et al.  Generalized pattern search algorithm for Peptide structure prediction. , 2008, Biophysical journal.

[6]  Sharif Rahman,et al.  A hybrid surrogate and pattern search optimization method and application to microelectronics , 2006 .

[7]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[8]  CHARLES AUDET,et al.  Finding Optimal Algorithmic Parameters Using Derivative-Free Optimization , 2006, SIAM J. Optim..

[9]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[10]  Virginia Torczon,et al.  Using approximations to accelerate engineering design optimization , 1998 .

[11]  Meng Wang,et al.  Optimal Aeroacoustic Shape Design Using the Surrogate Management Framework , 2003 .

[12]  Katya Scheinberg,et al.  On the local convergence of a derivative-free algorithm for least-squares minimization , 2010, Computational Optimization and Applications.

[13]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[14]  Humberto Rocha,et al.  Incorporating minimum Frobenius norm models in direct search , 2010, Comput. Optim. Appl..

[15]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[16]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[17]  Luís N. Vicente,et al.  A particle swarm pattern search method for bound constrained global optimization , 2007, J. Glob. Optim..

[18]  Charles Audet,et al.  Globalization strategies for Mesh Adaptive Direct Search , 2008, Comput. Optim. Appl..

[19]  Katya Scheinberg,et al.  Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization , 2012, Mathematical Programming.

[20]  John E. Dennis,et al.  A framework for managing models in nonlinear optimization of computationally expensive functions , 1999 .

[21]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[22]  Katya Scheinberg,et al.  Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization , 2010, SIAM J. Optim..