Convergence of Liu-Storey conjugate gradient method

Abstract The conjugate gradient method is a useful and powerful approach for solving large-scale minimization problems. Liu and Storey developed a conjugate gradient method, which has good numerical performance but no global convergence result under traditional line searches such as Armijo, Wolfe and Goldstein line searches. In this paper a convergent version of Liu–Storey conjugate gradient method (LS in short) is proposed for minimizing functions that have Lipschitz continuous partial derivatives. By estimating the Lipschitz constant of the derivative of objective functions, we can find an adequate step size at each iteration so as to guarantee the global convergence and improve the efficiency of LS method in practical computation.

[1]  Xiaoqi Yang,et al.  Quadratic cost flow and the conjugate gradient method , 2005, Eur. J. Oper. Res..

[2]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[3]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[4]  Jiapu Zhang,et al.  Global Convergence of Conjugate Gradient Methods without Line Search , 2001, Ann. Oper. Res..

[5]  Yu-Hong Dai.  New properties of a nonlinear conjugate gradient method , 2001, Numerische Mathematik.

[6]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[7]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[8]  Jie Sun,et al.  Global convergence of a two-parameter family of conjugate gradient methods without line search , 2002 .

[9]  Ya-Xiang Yuan,et al.  Convergence properties of the Fletcher-Reeves method , 1996 .

[10]  Ya-Xiang Yuan.  Analysis on the conjugate gradient method , 1993 .

[11]  Jie Shen,et al.  Convergence of descent method without line search , 2005, Appl. Math. Comput..

[12]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[13]  G. Fasano.  Planar Conjugate Gradient Algorithm for Large-Scale Unconstrained Optimization, Part 2: Application , 2005 .

[14]  Ya-Xiang Yuan,et al.  Convergence Properties of Nonlinear Conjugate Gradient Methods , 1999, SIAM J. Optim..

[15]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[16]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[17]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[18]  C. Storey,et al.  Global convergence result for conjugate gradient methods , 1991 .

[19]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[20]  G. Fasano.  Planar Conjugate Gradient Algorithm for Large-Scale Unconstrained Optimization, Part 1: Theory , 2005 .

[21]  Luigi Grippo,et al.  Convergence conditions, line search algorithms and trust region implementations for the Polak–Ribière conjugate gradient method , 2005, Optim. Methods Softw..

[22]  Zhen-Jun Shi,et al.  Step-size estimation for unconstrained optimization methods , 2005 .

[23]  Luigi Grippo,et al.  A globally convergent version of the Polak-Ribière conjugate gradient method , 1997, Math. Program..

[24]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[25]  David G. Wilson,et al.  2. Constrained Optimization , 2005 .