An ODE-based nonmonotone method for unconstrained optimization problems

This paper proposes an ODE-based nonmonotone method for unconstrained optimization problems, which combines the idea of IMPBOT with the nonmonotone technique. The main characteristic of this method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified L-BFGS two loop recursion that requires only vector inner products, thus reducing the matrix computation and storage. Then a modified nonmonotone line search is performed to generate next iterative point instead of resolving the linear system. Under some reasonable assumptions, the method is proven to be globally and superlinearly convergent. Numerical results show the efficiency of this proposed method in practical computation.

[1]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[2]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[3]  M. Bartholomew-Biggs,et al.  Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations , 1989 .

[4]  N. Deng,et al.  Nonmonotonic trust region algorithm , 1993 .

[5]  Philippe L. Toint,et al.  An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization , 1996, SIAM J. Sci. Comput..

[6]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[7]  Ya-Xiang Yuan.  Advances in Nonlinear Programming , 1998 .

[8]  Desmond J. Higham,et al.  Trust Region Algorithms and Timestep Selection , 1999, SIAM J. Numer. Anal..

[9]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[10]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[11]  Yuhong Dai.  On the Nonmonotone Line Search , 2002 .

[12]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[13]  Wenyu Sun,et al.  Nonmonotone trust region method for solving optimization problems , 2004, Appl. Math. Comput..

[14]  E. Michael Gertz,et al.  A quasi-Newton trust-region method , 2004, Math. Program..

[15]  Jie Shen,et al.  Convergence of descent method without line search , 2005, Appl. Math. Comput..

[16]  Zhen-Jun Shi,et al.  Convergence of nonmonotone line search method , 2006 .

[17]  Wenyu Sun,et al.  An unconstrained optimization method using nonmonotone second order Goldstein’s line search , 2007 .

[18]  Neng-zhu Gu,et al.  Incorporating nonmonotone strategies into the trust region method for unconstrained optimization , 2008, Comput. Math. Appl..

[19]  Fusheng Wang,et al.  A variant of trust-region methods for unconstrained optimization , 2008, Appl. Math. Comput..

[20]  L. Liao,et al.  Combining Trust-Region Techniques and Rosenbrock Methods to Compute Stationary Points , 2009 .

[21]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[22]  Yigui Ou,et al.  A hybrid trust region algorithm for unconstrained optimization , 2011 .

[23]  Yigui Ou.  An ODE-Based Trust Region Filter Algorithm for Unconstrained Optimization , 2011 .

[24]  Qunyan Zhou,et al.  A new nonmonotone adaptive trust region method based on simple quadratic models , 2012 .