Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization

It is very important to generate a descent search direction independent of line searches in showing the global convergence of conjugate gradient methods. The method of Hager and Zhang (2005) satisfies the sufficient descent condition. In this paper, we treat two subjects. We first consider a unified formula of parameters which establishes the sufficient descent condition and follows the modification technique of Hager and Zhang. In order to show the global convergence of the conjugate gradient method with the unified formula of parameters, we define some property (say Property A). We prove the global convergence of the method with Property A. Next, we apply the unified formula to a scaled conjugate gradient method and show its global convergence property. Finally numerical results are given.

[1]  Hiroshi Yabe,et al.  Conjugate gradient methods based on secant conditions that generate descent search directions for unconstrained optimization , 2012, J. Comput. Appl. Math..

[2]  Bo-Shi Tian,et al.  Global convergence of some modified PRP nonlinear conjugate gradient methods , 2011, Optim. Lett..

[3]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[4]  Neculai Andrei,et al.  A Dai-Yuan conjugate gradient algorithm with sufficient descent and conjugacy conditions for unconstrained optimization , 2008, Appl. Math. Lett..

[5]  Li Zhang,et al.  Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search , 2006, Numerische Mathematik.

[6]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[7]  Gonglin Yuan,et al.  Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems , 2009, Optim. Lett..

[8]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[9]  Neculai Andrei,et al.  New accelerated conjugate gradient algorithms as a modification of Dai-Yuan's computational scheme for unconstrained optimization , 2010, J. Comput. Appl. Math..

[10]  Li Zhang,et al.  A new globalization technique for nonlinear conjugate gradient methods for nonconvex minimization , 2011, Appl. Math. Comput..

[11]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[12]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[13]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[14]  Jie Sun,et al.  Global convergence of a two-parameter family of conjugate gradient methods without line search , 2002 .

[15]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[16]  Gaohang Yu,et al.  Global convergence of modified Polak-Ribière-Polyak conjugate gradient methods with sufficient descent property , 2008 .

[17]  Wufan Chen,et al.  Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization , 2008, Optim. Methods Softw..

[18]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[19]  Zhifeng Dai,et al.  A modified CG-DESCENT method for unconstrained optimization , 2011, J. Comput. Appl. Math..

[20]  Hiroshi Yabe,et al.  A Three-Term Conjugate Gradient Method with Sufficient Descent Property for Unconstrained Optimization , 2011, SIAM J. Optim..

[21]  W. Cheng.  A Two-Term PRP-Based Descent Method , 2007 .

[22]  Ya-Xiang Yuan,et al.  A three-parameter family of nonlinear conjugate gradient methods , 2001, Math. Comput..

[23]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[24]  Yuhong Dai.  Nonlinear Conjugate Gradient Methods , 2011 .

[25]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[26]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[27]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[28]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[29]  Hiroshi Yabe,et al.  Globally Convergent Three-Term Conjugate Gradient Methods that Use Secant Conditions and Generate Descent Search Directions for Unconstrained Optimization , 2011, Journal of Optimization Theory and Applications.

[30]  Min Li,et al.  A sufficient descent LS conjugate gradient method for unconstrained optimization problems , 2011, Appl. Math. Comput..