Nonmonotone adaptive trust region method

In this paper, we propose a nonmonotone adaptive trust region method for unconstrained optimization problems. This method can produce an adaptive trust region radius automatically at each iteration and allow the functional value of iterates to increase within finite iterations and finally decrease after such finite iterations. This nonmonotone approach and adaptive trust region radius can reduce the number of solving trust region subproblems when reaching the same precision. The global convergence and convergence rate of this method are analyzed under some mild conditions. Numerical results show that the proposed method is effective in practical computation.

[1]  Jinhua Guo,et al.  A new trust region method with adaptive radius , 2008, Comput. Optim. Appl..

[2]  Zhen-Jun Shi,et al.  Convergence of line search methods for unconstrained optimization , 2004, Appl. Math. Comput..

[3]  Zhen-Jun Shi,et al.  Convergence of quasi-Newton method with new inexact line search☆ , 2006 .

[4]  Luigi Grippo,et al.  Convergence conditions, line search algorithms and trust region implementations for the Polak–Ribière conjugate gradient method , 2005, Optim. Methods Softw..

[5]  M. J. D. Powell,et al.  On the global convergence of trust region algorithms for unconstrained minimization , 1984, Math. Program..

[6]  N. Deng,et al.  Nonmonotonic trust region algorithm , 1993 .

[7]  Michel Bierlaire,et al.  Dealing with singularities in nonlinear unconstrained optimization , 2009, European Journal of Operational Research.

[8]  Xiang-Sun Zhang,et al.  A nonmonotone adaptive trust region method and its convergence , 2003 .

[9]  Jin-yanFan,et al.  A LINE SEARCH AND TRUST REGION ALGORITHM WITH TRUST REGION RADIUS CONVERGING TO ZERO , 2004 .

[10]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[11]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[12]  Florian A. Potra,et al.  Efficient line search algorithm for unconstrained optimization , 1995 .

[13]  A LINE SEARCH AND TRUST REGION ALGORITHM WITH TRUST REGION RADIUS CONVERGING TO ZERO ∗1) , 2004 .

[14]  Boris T. Polyak,et al.  Newton's method and its use in optimization , 2007, Eur. J. Oper. Res..

[15]  Jie Shen,et al.  Convergence analysis of adaptive trust region methods , 2007, RAIRO Oper. Res..

[16]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[17]  Jérome M. B. Walmag,et al.  A Note on Trust-Region Radius Update , 2005, SIAM J. Optim..

[18]  Ya-Xiang Yuan,et al.  Optimization theory and methods , 2006 .

[19]  Desmond J. Higham,et al.  Trust Region Algorithms and Timestep Selection , 1999, SIAM J. Numer. Anal..

[20]  Qin Ni,et al.  Optimality Conditions for Trust-Region Subproblems Involving a Conic Model , 2005, SIAM J. Optim..

[21]  Yuhong Dai.  On the Nonmonotone Line Search , 2002 .

[22]  Zhen-Jun Shi,et al.  Convergence of nonmonotone line search method , 2006 .

[23]  Defeng Sun,et al.  Global convergece of the bfgs algorithm with nonmonotone linesearch ∗ ∗this work is supported by national natural science foundation$ef: , 1995 .

[24]  E. Michael Gertz,et al.  A quasi-Newton trust-region method , 2004, Math. Program..

[25]  LongHei.  A SELF—ADAPTIVE TRUST REGION ALGORITHM , 2003 .

[26]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[27]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .

[28]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[29]  Han Jiye.  A nonmonotone trust region algorithm for equality constrained optimization , 1995 .

[30]  Jorge Nocedal,et al.  Theory of algorithms for unconstrained optimization , 1992, Acta Numerica.

[31]  Zhenjun Shi,et al.  A gradient-related algorithm with inexact line searches , 2004 .

[32]  Li-Zhi Liao,et al.  An adaptive trust region method and its convergence , 2002 .

[33]  Neculai Andrei,et al.  Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2010, Eur. J. Oper. Res..

[34]  Annick Sartenaer,et al.  A Class of Trust Region Methods for Nonlinear Network Optimization Problems , 1995, SIAM J. Optim..

[35]  Ya-Xiang Yuan.  Advances in Nonlinear Programming , 1998 .

[36]  Josefa Mula,et al.  Mathematical programming models for supply chain production and transport planning , 2010, Eur. J. Oper. Res..

[37]  M. J. D. Powell,et al.  Direct search algorithms for optimization calculations , 1998, Acta Numerica.

[38]  Wenyu Sun,et al.  Nonmonotone adaptive trust-region method for unconstrained optimization problems , 2005, Appl. Math. Comput..

[39]  Z. J. Shi,et al.  New Inexact Line Search Method for Unconstrained Optimization , 2005 .

[41]  Neculai Andrei,et al.  Scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2007, Optim. Methods Softw..

[42]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..