A wedge trust region method with self-correcting geometry for derivative-free optimization

Recently, some methods for solving optimization problems without derivatives have been proposed. The main part of these methods is to form a suitable model function that can be minimized for obtaining a new iterative point. An important strategy is geometry-improving iteration for a good model, which needs a lot of calculations. Besides, Marazzi and Nocedal (2002) proposed a wedge trust region method for derivative free optimization. In this paper, we propose a new self-correcting geometry procedure with less computational efforts, and combine it with the wedge trust region method. The global convergence of new algorithm is established. The limited numerical experiments show that the new algorithm is efficient and competitive.

[1]  Jorge Nocedal,et al.  Wedge trust region methods for derivative free optimization , 2002, Math. Program..

[2]  Ya-Xiang Yuan,et al.  A trust region algorithm for equality constrained optimization , 1990, Math. Program..

[3]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[4]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[5]  M. Powell.  On the Lagrange functions of quadratic models that are defined by interpolation* , 2001 .

[6]  Ya-Xiang Yuan,et al.  On a subproblem of trust region algorithms for constrained optimization , 1990, Math. Program..

[7]  Wenyu Sun,et al.  On convergence analysis of a derivative-free trust region algorithm for constrained optimization with separable structure , 2014 .

[8]  Katya Scheinberg,et al.  On the local convergence of a derivative-free algorithm for least-squares minimization , 2010, Computational Optimization and Applications.

[9]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[10]  Ya-Xiang Yuan,et al.  A Robust Algorithm for Optimization with General Equality and Inequality Constraints , 2000, SIAM J. Sci. Comput..

[11]  Serge Gratton,et al.  An active-set trust-region method for derivative-free nonlinear bound-constrained optimization , 2011, Optim. Methods Softw..

[12]  M. J. D. Powell,et al.  Least Frobenius norm updating of quadratic models that satisfy interpolation conditions , 2004, Math. Program..

[13]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[14]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[15]  Charles Audet,et al.  Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems , 2008 .

[16]  Wenyu Sun,et al.  Nonmonotone retrospective conic trust region method for unconstrained optimization , 2013 .

[17]  R. Oeuvray.  Trust-region methods based on radial basis functions with application to biomedical imaging , 2005 .

[18]  G. R. Liu,et al.  I. Computational methods , 2006, Radiative Neutron Capture.

[19]  Wen-yuSun,et al.  CONIC TRUST REGION METHOD FOR LINEARLY CONSTRAINED OPTIMIZATION , 2003 .

[20]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[21]  Tamara G. Kolda,et al.  Optimizing an Empirical Scoring Function for Transmembrane Protein Structure Determination , 2004, INFORMS J. Comput..

[22]  W. Hager,et al.  The cyclic Barzilai-–Borwein method for unconstrained optimization , 2006 .

[23]  P. G. Ciarlet,et al.  General lagrange and hermite interpolation in Rn with applications to finite element methods , 1972 .

[24]  Ya-Xiang Yuan,et al.  A Conic Trust-Region Method for Nonlinearly Constrained Optimization , 2001, Ann. Oper. Res..

[25]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[26]  Katya Scheinberg,et al.  Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization , 2010, SIAM J. Optim..

[27]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[28]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[29]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[30]  M. J. D. Powell,et al.  On nonlinear optimization since 1959 , 2009, The Birth of Numerical Analysis.

[31]  Katya Scheinberg,et al.  A derivative free optimization algorithm in practice , 1998 .

[32]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[33]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[34]  M. J. D. Powell,et al.  On the global convergence of trust region algorithms for unconstrained minimization , 1984, Math. Program..

[35]  D. Winfield,et al.  Function Minimization by Interpolation in a Data Table , 1973 .

[36]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.