A globally convergent version of the Polak-Ribière conjugate gradient method

In this paper we propose a new line search algorithm that ensures global convergence of the Polak-Ribière conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions. In particular, we show that with this line search every limit point produced by the Polak-Ribière iteration is a stationary point of the objective function. Moreover, we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite Hessian matrix. Under strong convexity assumptions, the known global convergence results can be reobtained as a special case. From a computational point of view, we may expect that an algorithm incorporating the step-size acceptance rules proposed here will retain the same good features of the Polak-Ribière method, while avoiding pathological situations.

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[3]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[4]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[5]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[6]  Magnus R. Hestenes,et al.  Conjugate Direction Methods in Optimization , 1980 .

[7]  Luigi Grippo,et al.  Stopping criteria for linesearch methods without derivatives , 1984, Math. Program..

[8]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[9]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[10]  David F. Shanno.  Globally convergent conjugate gradient algorithms , 1985, Math. Program..

[11]  M. Powell.  Convergence properties of algorithms for nonlinear optimization , 1986 .

[12]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[13]  L. Grippo,et al.  Global convergence and stabilization of unconstrained minimization methods without derivatives , 1988 .

[14]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[15]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[16]  C. Storey,et al.  Generalized Polak-Ribière algorithm , 1992 .