On the sufficient descent condition of the Hager-Zhang conjugate gradient methods

Based on an eigenvalue study, the sufficient descent condition of an extended class of the Hager-Zhang nonlinear conjugate gradient methods is established. As an interesting result, it is shown that the search directions of the CG_Descent algorithm satisfy the sufficient descent condition $$d_k^Tg_k<-\frac{7}{8}||g_k||^2$$dkTgk<-78||gk||2.

[1]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[2]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[3]  Y. -H. Dai,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[4]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[5]  Ya-Xiang Yuan,et al.  Convergence Properties of Nonlinear Conjugate Gradient Methods , 1999, SIAM J. Optim..

[6]  Vidal Cohen,et al.  La recherche opérationnelle , 1995 .

[7]  Boris Polyak.  The conjugate gradient method in extreme problems , 2015 .

[8]  Yu-Hong Dai.  New properties of a nonlinear conjugate gradient method , 2001, Numerische Mathematik.

[9]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[10]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[11]  Saman Babaie-Kafaki,et al.  A note on the global convergence theorem of the scaled conjugate gradient algorithms proposed by Andrei , 2012, Comput. Optim. Appl..

[12]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[13]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[14]  William W. Hager,et al.  Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent , 2006, TOMS.

[15]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[16]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[17]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[18]  Yu-Hong Dai,et al.  A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search , 2013, SIAM J. Optim..