On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization

A nonlinear stepsize control framework for unconstrained optimization was recently proposed by Toint (Optim Methods Softw 28:82–95, 2013), providing a unified setting in which the global convergence can be proved for trust-region algorithms and regularization schemes. The original analysis assumes that the Hessians of the models are uniformly bounded. In this paper, the global convergence of the nonlinear stepsize control algorithm is proved under the assumption that the norm of the Hessians can grow by a constant amount at each iteration. The worst-case complexity is also investigated. The results obtained for unconstrained smooth optimization are extended to some algorithms for composite nonsmooth optimization and unconstrained multiobjective optimization as well.

[1]  P. Toint.  Global Convergence of a a of Trust-Region Methods for Nonconvex Minimization in Hilbert Space , 1988 .

[2]  Ya-Xiang Yuan,et al.  Conditions for convergence of trust region algorithms for nonsmooth optimization , 1985, Math. Program..

[3]  Y. Nesterov.  Modified Gauss-Newton Scheme with Worst-Case Guarantees for its Global Performance , 2003 .

[4]  YuhongDai,et al.  A NEW FAMILY OF TRUST ERGION ALGORITHMS FOR UNCONSTRAINED OPTIMIZATION , 2003 .

[5]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[6]  Nicholas I. M. Gould,et al.  On the Evaluation Complexity of Composite Function Minimization with Applications to Nonconvex Nonlinear Programming , 2011, SIAM J. Optim..

[7]  Nicholas I. M. Gould,et al.  Convergence of a Regularized Euclidean Residual Algorithm for Nonlinear Least-Squares , 2010, SIAM J. Numer. Anal..

[8]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[9]  Jinyan Fan.  Convergence Rate of The Trust Region Method for Nonlinear Equations Under Local Error Bound Condition , 2006, Comput. Optim. Appl..

[10]  Nan-Jing Huang,et al.  Optimality Conditions for Vector Optimization Problems , 2009, J. Optimization Theory and Applications.

[11]  Yong Wang,et al.  A new trust region method for nonlinear equations , 2003, Math. Methods Oper. Res..

[12]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity , 2011, Math. Program..

[13]  Philippe L. Toint.  Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization , 2013, Optim. Methods Softw..

[14]  Ya-Xiang Yuan,et al.  On the superlinear convergence of a trust region algorithm for nonsmooth optimization , 1985, Math. Program..

[15]  D. T. Luc,et al.  Optimality Conditions for C1,1 Vector Optimization Problems , 2001 .

[16]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2011, Math. Program..

[17]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[18]  Jörg Fliege,et al.  Steepest descent methods for multicriteria optimization , 2000, Math. Methods Oper. Res..

[19]  R. Fletcher.  A model algorithm for composite nondifferentiable optimization problems , 1982 .

[20]  Antoine Soubeyran,et al.  A Trust-Region Method for Unconstrained Multiobjective Problems with Applications in Satisficing Processes , 2014, J. Optim. Theory Appl..

[21]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[22]  M. J. D. Powell,et al.  On the global convergence of trust region algorithms for unconstrained minimization , 1984, Math. Program..

[23]  Yurii Nesterov,et al.  Modified Gauss–Newton scheme with worst case guarantees for global performance , 2007, Optim. Methods Softw..

[24]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .