Strong Evaluation Complexity of An Inexact Trust-Region Algorithm for Arbitrary-Order Unconstrained Nonconvex Optimization.

A trust-region algorithm using inexact function and derivatives values is introduced for solving unconstrained smooth optimization problems. This algorithm uses high-order Taylor models and allows the search of strong approximate minimizers of arbitrary order. The evaluation complexity of finding a $q$-th approximate minimizer using this algorithm is then shown, under standard conditions, to be $\mathcal{O}\big(\min_{j\in\{1,\ldots,q\}}\epsilon_j^{-(q+1)}\big)$ where the $\epsilon_j$ are the order-dependent requested accuracy thresholds. Remarkably, this order is identical to that of classical trust-region methods using exact information.

[1]  W. Hogan.  Point-to-Set Maps in Mathematical Programming , 1973 .

[2]  D. Winfield,et al.  Function Minimization by Interpolation in a Data Table , 1973 .

[3]  Katya Scheinberg,et al.  On the convergence of derivative-free methods for unconstrained optimization , 1997 .

[4]  Valeria Simoncini,et al.  Theory of Inexact Krylov Subspace Methods and Applications to Scientific Computing , 2003, SIAM J. Sci. Comput..

[5]  Gerard L. G. Sleijpen,et al.  Inexact Krylov Subspace Methods for Linear Systems , 2004, SIAM J. Matrix Anal. Appl..

[6]  Randall J. LeVeque,et al.  Finite difference methods for ordinary and partial differential equations - steady-state and time-dependent problems , 2007 .

[7]  John L. Nazareth,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[8]  Nicholas I. M. Gould,et al.  On the Oracle Complexity of First-Order and Derivative-Free Algorithms for Smooth Nonconvex Minimization , 2012, SIAM J. Optim..

[9]  José Mario Martínez,et al.  Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models , 2017, Math. Program..

[10]  Serge Gratton,et al.  Minimizing convex quadratic with variable precision Krylov methods , 2018, ArXiv.

[11]  S. Bellavia,et al.  Adaptive Regularization Algorithms with Inexact Evaluations for Nonconvex Optimization , 2018, SIAM J. Optim..

[12]  Nicholas I. M. Gould,et al.  Sharp worst-case evaluation complexity bounds for arbitrary-order nonconvex optimization with inexpensive constraints , 2018, SIAM J. Optim..

[13]  José Mario Martínez,et al.  On the use of third-order models with fourth-order regularization for unconstrained optimization , 2019, Optimization Letters.

[14]  P. Toint,et al.  Strong Evaluation Complexity Bounds for Arbitrary-Order Optimization of Nonconvex Nonsmooth Composite Functions , 2020, 2001.10802.

[15]  High-order Evaluation Complexity of a Stochastic Adaptive Regularization Algorithm for Nonconvex Optimization Using Inexact Function Evaluations and Randomly Perturbed Derivatives , 2020, 2005.04639.

[16]  E. Simon,et al.  An algorithm for the minimization of nonsmooth nonconvex functions using inexact evaluations and its worst-case complexity , 2019, Math. Program..