Another hybrid conjugate gradient algorithm for unconstrained optimization

Another hybrid conjugate gradient algorithm is subject to analysis. The parameter βk is computed as a convex combination of $$ \beta ^{{HS}}_{k} $$ (Hestenes-Stiefel) and $$ \beta ^{{DY}}_{k} $$ (Dai-Yuan) algorithms, i.e. $$ \beta ^{C}_{k} = {\left( {1 - \theta _{k} } \right)}\beta ^{{HS}}_{k} + \theta _{k} \beta ^{{DY}}_{k} $$. The parameter θk in the convex combination is computed in such a way so that the direction corresponding to the conjugate gradient algorithm to be the Newton direction and the pair (sk, yk) to satisfy the quasi-Newton equation $$ \nabla ^{2} f{\left( {x_{{k + 1}} } \right)}s_{k} = y_{k} $$, where $$ s_{k} = x_{{k + 1}} - x_{k} $$ and $$ y_{k} = g_{{k + 1}} - g_{k} $$. The algorithm uses the standard Wolfe line search conditions. Numerical comparisons with conjugate gradient algorithms show that this hybrid computational scheme outperforms the Hestenes-Stiefel and the Dai-Yuan conjugate gradient algorithms as well as the hybrid conjugate gradient algorithms of Dai and Yuan. A set of 750 unconstrained optimization problems are used, some of them from the CUTE library.

[1]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[2]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[3]  R. Fletcher Practical Methods of Optimization , 1988 .

[4]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[5]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[6]  Neculai Andrei,et al.  A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2007, Appl. Math. Lett..

[7]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[8]  Yu-Hong Dai New properties of a nonlinear conjugate gradient method , 2001, Numerische Mathematik.

[9]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[10]  Ya-Xiang Yuan,et al.  Convergence Properties of Nonlinear Conjugate Gradient Methods , 1999, SIAM J. Optim..

[11]  Boris Polyak The conjugate gradient method in extremal problems , 1969 .

[12]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[13]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[14]  Ya-Xiang Yuan,et al.  An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization , 2001, Ann. Oper. Res..

[15]  M. Powell Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[16]  Neculai Andrei,et al.  Scaled conjugate gradient algorithms for unconstrained optimization , 2007, Comput. Optim. Appl..

[17]  C. Storey,et al.  Global convergence result for conjugate gradient methods , 1991 .

[18]  D. Touati-Ahmed,et al.  Efficient hybrid conjugate gradient techniques , 1990 .

[19]  Neculai Andrei,et al.  Scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2007, Optim. Methods Softw..

[20]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[21]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[22]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[23]  Jingfeng Zhang,et al.  New Quasi-Newton Equation and Related Methods for Unconstrained Optimization , 1999 .

[24]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[25]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .