Improved worst-case evaluation complexity for potentially rank-deficient nonlinear least-Euclidean-norm problems using higher-order regularized models

Given a sufficiently smooth vector-valued function r(x), a local minimizer of ‖r(x)‖2 within a closed, non-empty, convex set F is sought by modelling ‖r(x)‖q2/q with a p-th order Taylorseries approximation plus a (p + 1)-st order regularization term for given even p and some appropriate associated q. The resulting algorithm is guaranteed to find a value x̄ for which ‖r(x̄)‖2 ≤ ǫp or χ(x̄) ≤ ǫd, for some first-order criticality measure χ(x) of ‖r(x)‖2 within F , using at most O(max{max(ǫd, χmin) ,max(ǫp, rmin) −1/2i}) evaluations of r(x) and its derivatives; here rmin and χmin ≥ 0 are any lower bounds on ‖r(x)‖2 and χ(x), respectively, and 2 is the highest power of 2 that divides p. An improved bound is possible under a suitable full-rank assumption.

[1]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[2]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[3]  Nicholas I. M. Gould,et al.  On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems , 2010, SIAM J. Optim..

[4]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity , 2011, Math. Program..

[5]  P. Toint,et al.  An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity , 2012 .

[6]  Nicholas I. M. Gould,et al.  On the Evaluation Complexity of Cubic Regularization Methods for Potentially Rank-Deficient Nonlinear Least-Squares Problems and Its Relevance to Constrained Nonlinear Optimization , 2013, SIAM J. Optim..

[7]  José Mario Martínez,et al.  Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models , 2017, Math. Program..

[8]  Daniel P. Robinson,et al.  A trust region algorithm with a worst-case iteration complexity of O(ϵ-3/2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{docume , 2016, Mathematical Programming.

[9]  P. Toint,et al.  Evaluation Complexity Bounds for Smooth Constrained Nonlinear Optimization Using Scaled KKT Conditions and High-Order Models , 2019, Approximation and Optimization.