A family of three-term conjugate gradient methods with sufficient descent property for unconstrained optimization

Recently, conjugate gradient methods, which usually generate descent search directions, are useful for large-scale optimization. Narushima et al. (SIAM J Optim 21:212–230, 2011) have proposed a three-term conjugate gradient method which satisfies a sufficient descent condition. We extend this method to two parameters family of three-term conjugate gradient methods which can be used to control the magnitude of the directional derivative. We show that these methods converge globally and work well for suitable choices of the parameters. Numerical results are also presented.

[1]  Li Zhang,et al.  Some descent three-term conjugate gradient methods and their global convergence , 2007, Optim. Methods Softw..

[2]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[3]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[4]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  Hiroshi Yabe,et al.  Globally Convergent Three-Term Conjugate Gradient Methods that Use Secant Conditions and Generate Descent Search Directions for Unconstrained Optimization , 2011, Journal of Optimization Theory and Applications.

[7]  Li Zhang.  A new Liu-Storey type nonlinear conjugate gradient method for unconstrained optimization problems , 2009 .

[8]  Issam A. R. Moghrabi,et al.  Alternative parameter choices for multi-step Quasi-Newton methods , 1993 .

[9]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[10]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[11]  Issam A. R. Moghrabi,et al.  Multi-step quasi-Newton methods for optimization , 1994 .

[12]  Li Zhang,et al.  A nonlinear conjugate gradient method based on the MBFGS secant condition , 2006, Optim. Methods Softw..

[13]  Li Zhang,et al.  Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search , 2006, Numerische Mathematik.

[14]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .

[15]  Hiroshi Yabe,et al.  Global Convergence Properties of Nonlinear Conjugate Gradient Methods with Modified Secant Condition , 2004, Comput. Optim. Appl..

[16]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[17]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[18]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[19]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[20]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[21]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[22]  H. Sorenson.  Comparison of some conjugate direction procedures for function minimization , 1969 .

[23]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[24]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[25]  William W. Hager,et al.  The Limited Memory Conjugate Gradient Method , 2013, SIAM J. Optim..

[26]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[27]  Gaohang Yu,et al.  Global convergence of modified Polak-Ribière-Polyak conjugate gradient methods with sufficient descent property , 2008 .

[28]  Hiroshi Yabe,et al.  A new nonlinear conjugate gradient method for unconstrained optimization , 2005 .

[29]  Yu-Hong Dai,et al.  A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search , 2013, SIAM J. Optim..

[30]  William W. Hager,et al.  Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent , 2006, TOMS.

[31]  Hiroshi Yabe,et al.  A Three-Term Conjugate Gradient Method with Sufficient Descent Property for Unconstrained Optimization , 2011, SIAM J. Optim..

[32]  W. Cheng.  A Two-Term PRP-Based Descent Method , 2007 .

[33]  Hiroshi Yabe,et al.  Multi-step nonlinear conjugate gradient methods for unconstrained minimization , 2008, Comput. Optim. Appl..