Another nonlinear conjugate gradient algorithm for unconstrained optimization

A nonlinear conjugate gradient algorithm which is a modification of the Dai and Yuan [A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim. 10 (1999), pp. 177–182] conjugate gradient algorithm satisfying a parameterized sufficient descent condition with a parameter δ k is proposed. The parameter δ k is computed by means of the conjugacy condition, thus an algorithm which is a positive multiplicative modification of the Hestenes and Stiefel [Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards Sec. B 48 (1952), pp. 409–436] algorithm is obtained. The algorithm can be viewed as an adaptive version of the Dai and Liao [New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001), pp. 87–101] conjugate gradient algorithm. Close to our computational scheme is the conjugate gradient algorithm recently proposed by Hager and Zhang [A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim. 16 (2005), pp. 170–192]. Computational results, for a set consisting of 750 unconstrained optimization test problems, show that this new conjugate gradient algorithm substantially outperforms the known conjugate gradient algorithms.

[1]  C. Storey,et al.  Global convergence result for conjugate gradient methods , 1991 .

[2]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[3]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[4]  Yu-hongDai,et al.  TESTING DIFFERENT CONJUGATE GRADIENT METHODS FOR LARGE—SCALE UNCONSTRAINED OPTIMIZATION , 2003 .

[5]  Wufan Chen,et al.  Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization , 2008, Optim. Methods Softw..

[6]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[7]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[8]  Ya-Xiang Yuan,et al.  An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization , 2001, Ann. Oper. Res..

[9]  Neculai Andrei,et al.  Scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2007, Optim. Methods Softw..

[10]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[11]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[12]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[13]  Ya-Xiang Yuan,et al.  Convergence Properties of Nonlinear Conjugate Gradient Methods , 1999, SIAM J. Optim..

[14]  A. Perry.  A Class of Conjugate Gradient Algorithms with a Two-Step Variable Metric Memory , 1977 .

[15]  A. Iserles,et al.  Acta numerica 1992 , edited by A. Iserles. Pp. 407. £19.95 1992. ISBN 0-521-41026-6 (Cambridge University Press) , 1993, The Mathematical Gazette.

[16]  J. L. Nazareth,et al.  Linear and nonlinear conjugate gradient-related methods , 1996 .

[17]  Neculai Andrei,et al.  Scaled conjugate gradient algorithms for unconstrained optimization , 2007, Comput. Optim. Appl..

[18]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[19]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[20]  Neculai Andrei,et al.  A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2007, Appl. Math. Lett..

[21]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[22]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[23]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[24]  Dianne P. O'Leary,et al.  Conjugate Gradients and Related KMP Algorithms: The Beginnings , 1998 .

[25]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[26]  Jorge Nocedal,et al.  Theory of algorithms for unconstrained optimization , 1992, Acta Numerica.

[27]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[28]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[29]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[30]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[31]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[32]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[33]  D. Touati-Ahmed,et al.  Efficient hybrid conjugate gradient techniques , 1990 .

[34]  Gene H. Golub,et al.  Some History of the Conjugate Gradient and Lanczos Algorithms: 1948-1976 , 1989, SIAM Rev..

[35]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[36]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .