An inexact line search approach using modified nonmonotone strategy for unconstrained optimization

This paper concerns with a new nonmonotone strategy and its application to the line search approach for unconstrained optimization. It has been believed that nonmonotone techniques can improve the possibility of finding the global optimum and increase the convergence rate of the algorithms. We first introduce a new nonmonotone strategy which includes a convex combination of the maximum function value of some preceding successful iterates and the current function value. We then incorporate the proposed nonmonotone strategy into an inexact Armijo-type line search approach to construct a more relaxed line search procedure. The global convergence to first-order stationary points is subsequently proved and the R-linear convergence rate are established under suitable assumptions. Preliminary numerical results finally show the efficiency and the robustness of the proposed approach for solving unconstrained nonlinear optimization problems.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  Masoud Ahookhosh,et al.  An efficient nonmonotone trust-region method for unconstrained optimization , 2011, Numerical Algorithms.

[3]  Yuhong Dai.  On the Nonmonotone Line Search , 2002 .

[4]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[5]  Zhen-Jun Shi,et al.  Convergence of nonmonotone line search method , 2006 .

[6]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[7]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[8]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[9]  Masoud Ahookhosh,et al.  Computers and Mathematics with Applications a Nonmonotone Trust Region Method with Adaptive Radius for Unconstrained Optimization Problems , 2022 .

[10]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[11]  A. Tits,et al.  Avoiding the Maratos effect by means of a nonmonotone line search I. general constrained problems , 1991 .

[12]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[13]  C. Lemaréchal,et al.  The watchdog technique for forcing convergence in algorithms for constrained optimization , 1982 .

[14]  N. Andrei.  Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization , 2009 .

[15]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[16]  Shengquan Wang,et al.  Modified nonmonotone Armijo line search for descent method , 2010, Numerical Algorithms.

[17]  L. Grippo,et al.  A class of nonmonotone stabilization methods in unconstrained optimization , 1991 .

[18]  Philippe L. Toint,et al.  An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization , 1996, SIAM J. Sci. Comput..

[19]  L. Grippo,et al.  A truncated Newton method with nonmonotone line search for unconstrained optimization , 1989 .

[20]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[21]  Saman Babaie-Kafaki,et al.  A modified two-point stepsize gradient algorithm for unconstrained minimization , 2013, Optim. Methods Softw..

[22]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[23]  Zheng-Hai Huang,et al.  A Non-monotone Line Search Algorithm for Unconstrained Optimization , 2010, J. Sci. Comput..

[24]  Masoud Ahookhosh,et al.  A nonmonotone trust-region line search method for large-scale unconstrained optimization , 2012 .

[25]  Masoud Ahookhosh,et al.  Optimization: a Journal of Mathematical Programming and Operations Research a Class of Nonmonotone Armijo-type Line Search Method for Unconstrained Optimization a Class of Nonmonotone Armijo-type Line Search Method for Unconstrained Optimization , 2022 .