论文信息 - Adaptive cubic overestimation methods for unconstrained optimization . Part II : worst-case iteration complexity

Adaptive cubic overestimation methods for unconstrained optimization . Part II : worst-case iteration complexity

An Adaptive Cubic Overestimation (ACO) framework for unconstrained optimization was proposed and analysed in Cartis, Gould & Toint (Part I, 2007). In this companion paper, we further the analysis by providing worst-case global iteration complexity bounds for ACO and a second-order variant to achieve approximate first-order, and for the latter even second-order, criticality of the iterates. In particular, the second-order ACO algorithm requires at mostO(ǫ) iterations to drive the objective’s gradient below the desired accuracy ǫ, and O(ǫ), to reach approximate nonnegative curvature in a subspace. The orders of these bounds match those proved by Nesterov & Polyak (Math. Programming 108(1), 2006, pp 177-205) for their Algorithm 3.3 which minimizes the cubic model globally on each iteration. Our approach is more general, and relevant to practical (large-scale) calculations, as ACO allows the cubic model to be solved only approximately and may employ approximate Hessians.

P. Toint | N. Gould | C. Cartis

[1] J. J. Moré,et al. A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[2] Gene H. Golub,et al. Matrix computations , 1983 .

[3] John E. Dennis,et al. Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[4] Nicholas I. M. Gould,et al. Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[5] Peter Deuflhard,et al. Newton Methods for Nonlinear Problems , 2004 .

[6] Yurii Nesterov,et al. Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[7] Yurii Nesterov,et al. Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[8] Peter Deuflhard,et al. Affine conjugate adaptive Newton methods for nonlinear elastomechanics , 2007, Optim. Methods Softw..

[9] P. Toint,et al. A recursive trust-region method in infinity norm for bound-constrained nonlinear optimization , 2007 .

[10] Serge Gratton,et al. Recursive Trust-Region Methods for Multiscale Nonlinear Optimization , 2008, SIAM J. Optim..

[11] Yurii Nesterov,et al. Accelerating the cubic regularization of Newton’s method on convex problems , 2005, Math. Program..

[12] P. Toint,et al. Adaptive cubic overestimation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2008 .