New properties of a nonlinear conjugate gradient method

Summary. This paper provides several new properties of the nonlinear conjugate gradient method in [5]. Firstly, the method is proved to have a certain self-adjusting property that is independent of the line search and the function convexity. Secondly, under mild assumptions on the objective function, the method is shown to be globally convergent with a variety of line searches. Thirdly, we find that instead of the negative gradient direction, the search direction defined by the nonlinear conjugate gradient method in [5] can be used to restart any optimization method while guaranteeing the global convergence of the method. Some numerical results are also presented.

[1]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[2]  Luigi Grippo,et al.  Stopping criteria for linesearch methods without derivatives , 1984, Math. Program..

[3]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[4]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[7]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[8]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[9]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[10]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[11]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[12]  Yu-Hong Dai,et al.  Some Properties of A New Conjugate Gradient Method , 1998 .

[13]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[14]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[15]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[16]  L. Grippo,et al.  Global convergence and stabilization of unconstrained minimization methods without derivatives , 1988 .

[17]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .