An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity

The adaptive cubic overestimation algorithm described in Cartis, Gould and Toint (2007) is adapted to the problem of minimizing a nonlinear, possibly nonconvex, smooth objective function over a convex domain. Convergence to first-order critical points is shown under standard assumptions, but without any Lipschitz continuity requirement on the objective’s Hessian. A worst-case complexity analysis in terms of evaluations of the problem’s function and derivatives is also presented for the Lipschitz continuous case and for a variant of the resulting algorithm. This analysis extends the best known bound for general unconstrained problems to nonlinear problems with convex constraints.

[1]  J. B. Rosen,et al.  Inequalities for Stochastic Nonlinear Programming Problems , 1964 .

[2]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[3]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[4]  S. Vavasis.  Nonlinear optimization: complexity issues , 1991 .

[5]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[6]  Nicholas I. M. Gould,et al.  Global Convergence of a Class of Trust Region Algorithms for Optimization Using Inexact Projections on Convex Constraints , 1993, SIAM J. Optim..

[7]  Stephen A. Vavasis,et al.  Black-Box Complexity of Local Minimization , 1993, SIAM J. Optim..

[8]  Annick Sartenaer,et al.  Armijo-type condition for the determination of a generalized Cauchy point in trust region algorithms using exact or inexact projections on convex constraints , 1993 .

[9]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[10]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[11]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[12]  Yurii Nesterov.  Cubic Regularization of Newton's Method for Convex Problems with Constraints , 2006 .

[13]  Peter Deuflhard,et al.  Affine conjugate adaptive Newton methods for nonlinear elastomechanics , 2007, Optim. Methods Softw..

[14]  P. Toint,et al.  A recursive trust-region method in infinity norm for bound-constrained nonlinear optimization , 2007 .

[15]  P. Toint,et al.  Adaptive cubic overestimation methods for unconstrained optimization , 2007 .

[16]  Serge Gratton,et al.  Recursive Trust-Region Methods for Multiscale Nonlinear Optimization , 2008, SIAM J. Optim..

[17]  P. Toint,et al.  A recursive ℓ∞-trust-region method for bound-constrained nonlinear optimization , 2008 .

[18]  Nicholas I. M. Gould,et al.  On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems , 2010, SIAM J. Optim..

[19]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2011, Math. Program..

[20]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity , 2011, Math. Program..

[21]  Ying Xiong.  Nonlinear Optimization , 2014 .