MNH: A Derivative-Free Optimization Algorithm Using Minimal Norm Hessians

We introduce MNH, a new algorithm for unconstrained optimization when derivatives are unavailable, primarily targeting applications that require running computationally expensive deterministic simulations. MNH relies on a trust-region framework with an underdetermined quadratic model that interpolates the function at a set of data points. We show how to construct this interpolation set to yield computationally stable parameters for the model and, in doing so, obtain an algorithm which converges to first-order critical points. Preliminary results are encouraging and show that MNH makes effective use of the points evaluated in the course of the optimization.

[1]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[2]  Katya Scheinberg,et al.  Recent progress in unconstrained nonlinear optimization without derivatives , 1997, Math. Program..

[3]  Tamara G. Kolda,et al.  Asynchronous parallel pattern search for nonlinear optimization , 2000 .

[4]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[5]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[6]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[7]  M. J. D. Powell,et al.  Least Frobenius norm updating of quadratic models that satisfy interpolation conditions , 2004, Math. Program..

[8]  R. Oeuvray.  Trust-region methods based on radial basis functions with application to biomedical imaging , 2005 .

[9]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[10]  Katya Scheinberg,et al.  Geometry of interpolation sets in derivative free optimization , 2007, Math. Program..

[11]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[12]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[13]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..