A family of three-term nonlinear conjugate gradient methods close to the memoryless BFGS method

Based on the memoryless BFGS quasi-Newton method, a family of three-term nonlinear conjugate gradient methods are proposed. For any line search, the directions generated by the new methods are sufficient descent. Using some efficient techniques, global convergence results are established when the line search fulfills the Wolfe or the Armijo conditions. Moreover, the r-linear convergence rate of the methods are analyzed as well. Numerical comparisons show that the proposed methods are efficient for the unconstrained optimization problems in the CUTEr library.

[1]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[2]  Reza Ghanbari,et al.  A descent family of Dai–Liao conjugate gradient methods , 2014, Optim. Methods Softw..

[3]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[4]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[5]  Jichun Li,et al.  Global pointwise error estimates for uniformly convergent finite element methods for the elliptic boundary layer problem , 1998 .

[6]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[7]  Yu-Hong Dai,et al.  A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search , 2013, SIAM J. Optim..

[8]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[9]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[10]  Li Zhang.  New versions of the Hestenes-Stiefel nonlinear conjugate gradient method based on the secant condition for optimization , 2009 .

[11]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[12]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .

[13]  Guoyin Li,et al.  New conjugacy condition and related new conjugate gradient methods for unconstrained optimization , 2007 .

[14]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[15]  Gonglin Yuan,et al.  Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems , 2009, Optim. Lett..

[16]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[17]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[18]  A. Perry.  A Class of Conjugate Gradient Algorithms with a Two-Step Variable Metric Memory , 1977 .

[19]  Heying Feng,et al.  The global convergence of a descent PRP conjugate gradient method , 2012 .

[20]  Hiroshi Yabe,et al.  Global Convergence Properties of Nonlinear Conjugate Gradient Methods with Modified Secant Condition , 2004, Comput. Optim. Appl..

[21]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[22]  Jian Zhang,et al.  A variant smoothing Newton method for P0-NCP based on a new smoothing function , 2009 .

[23]  Min Li,et al.  A sufficient descent LS conjugate gradient method for unconstrained optimization problems , 2011, Appl. Math. Comput..

[24]  Hassan A. Zedan.  New approach for tanh and extended-tanh methods with applications on Hirota-Satsuma equations , 2009 .

[25]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[26]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[27]  Neculai Andrei,et al.  Open Problems in Nonlinear Conjugate Gradient Algorithms for Unconstrained Optimization , 2011 .

[28]  M. Powell.  Convergence properties of algorithms for nonlinear optimization , 1986 .

[29]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[30]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[31]  L Guan.  Modified PRP Methods with Sufficient Descent Property and Their Convergence Properties , 2006 .

[32]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[33]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[34]  Jingfeng Zhang,et al.  New Quasi-Newton Equation and Related Methods for Unconstrained Optimization , 1999 .

[35]  Zhi-feng Dai,et al.  Two modified HS type conjugate gradient methods for unconstrained optimization problems , 2011 .

[36]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .