Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models

The worst-case evaluation complexity for smooth (possibly nonconvex) unconstrained optimization is considered. It is shown that, if one is willing to use derivatives of the objective function up to order p (for $$p\ge 1$$p≥1) and to assume Lipschitz continuity of the p-th derivative, then an $$\epsilon $$ϵ-approximate first-order critical point can be computed in at most $$O(\epsilon ^{-(p+1)/p})$$O(ϵ-(p+1)/p) evaluations of the problem’s objective function and its derivatives. This generalizes and subsumes results known for $$p=1$$p=1 and $$p=2$$p=2.

[1]  Stephen A. Vavasis,et al.  Black-Box Complexity of Local Minimization , 1993, SIAM J. Optim..

[2]  Y. Nesterov Modified Gauss-Newton Scheme with Worst-Case Guarantees for its Global Performance , 2003 .

[3]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[4]  Yurii Nesterov,et al.  Modified Gauss–Newton scheme with worst case guarantees for global performance , 2007, Optim. Methods Softw..

[5]  P. Toint,et al.  Adaptive cubic overestimation methods for unconstrained optimization , 2007 .

[6]  Serge Gratton,et al.  Recursive Trust-Region Methods for Multiscale Nonlinear Optimization , 2008, SIAM J. Optim..

[7]  P. Toint,et al.  Adaptive cubic overestimation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2008 .

[8]  Nicholas I. M. Gould,et al.  On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems , 2010, SIAM J. Optim..

[9]  Nicholas I. M. Gould,et al.  Convergence of a Regularized Euclidean Residual Algorithm for Nonlinear Least-Squares , 2010, SIAM J. Numer. Anal..

[10]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2011, Math. Program..

[11]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity , 2011, Math. Program..

[12]  P. Toint,et al.  An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity , 2012 .

[13]  Nicholas I. M. Gould,et al.  Evaluation complexity of adaptive cubic regularization methods for convex unconstrained optimization , 2012, Optim. Methods Softw..

[14]  Nicholas I. M. Gould,et al.  Complexity bounds for second-order optimality in unconstrained optimization , 2012, J. Complex..

[15]  Luís Nunes Vicente,et al.  Worst case complexity of direct search , 2013, EURO J. Comput. Optim..

[16]  Xiaojun Chen,et al.  Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization , 2015, Math. Program..

[17]  Ya-xiang Yuan,et al.  On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization , 2015, Math. Program..

[18]  J. Dussault Simple unified convergence proofs for Trust Region and a new ARC variant , 2015 .

[19]  Nicholas I. M. Gould,et al.  Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization , 2018, Found. Comput. Math..