ARCq: a new adaptive regularization by cubics

We present a new adaptive regularization by cubics (ARC) algorithm, which we name ARC and which is very close in spirit to trust region methods, closer than the original ARC is. We prove global convergence to second-order critical points. We also obtain as a corollary the convergence of the original ARC method. We prove the optimal complexity property for the ARC and identify the key elements which allow it. We end by proposing an efficient implementation using a Cholesky like factorization. Limited preliminary experimentation suggests that ARC may be more robust than its trust region counterpart and that our implementation is efficient.

[1]  Serge Gratton,et al.  Recursive Trust-Region Methods for Multiscale Nonlinear Optimization , 2008, SIAM J. Optim..

[2]  S. H. Cheng,et al.  A Modified Cholesky Algorithm Based on a Symmetric Indefinite Factorization , 1998, SIAM J. Matrix Anal. Appl..

[3]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[4]  Philippe L. Toint.  Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization , 2013, Optim. Methods Softw..

[5]  Nicholas I. M. Gould,et al.  On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems , 2010, SIAM J. Optim..

[6]  Iain Dunning,et al.  Computing in Operations Research Using Julia , 2013, INFORMS J. Comput..

[7]  J. Nocedal,et al.  The modified absolute-value factorization norm for trust-region minimization , 1998 .

[8]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[9]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[10]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[11]  J. Ortega.  Numerical Analysis: A Second Course , 1974 .

[12]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity , 2011, Math. Program..

[13]  Ya-xiang Yuan,et al.  On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization , 2015, Math. Program..

[14]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[15]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2011, Math. Program..

[16]  José Mario Martínez,et al.  Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models , 2017, Math. Program..

[17]  Daniel P. Robinson,et al.  A trust region algorithm with a worst-case iteration complexity of O(ϵ-3/2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{docume , 2016, Mathematical Programming.