A New Simple Conjugate Gradient Coefficient for Unconstrained Optimization

Abstract Conjugate gradient method holds an important role in solving unconstrained optimizations, especially for large scale problems. Numerous studies and modifications have been done to improve this method. In this paper, we propose a fundamentally different conjugate gradient method in which the well known coefficient β k is computed using the eigenvalues generated by using exact Hessian matrix of f ( x ) . This relatively easy formula for β k has made conjugate gradient method very simple, but still possesses global convergence properties. Numerical results have shown that this formula performs better than the original conjugate gradient methods. Mathematics Subject Classification : 65K10 Keywords : conjugate gradient methods, conjugate gradient coefficient, eigenvalues, Hessian matrix, global convergence properties 1. Introduction The conjugate gradient method (CG) plays an important role in solving the unconstrained optimization problem. In general, the method has the following form min f (

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[3]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[4]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[5]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[6]  K. E. Hillstrom,et al.  A Simulation Test Approach to the Evaluation of Nonlinear Optimization Algorithms , 1977, TOMS.

[7]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[8]  D. Touati-Ahmed,et al.  Efficient hybrid conjugate gradient techniques , 1990 .

[9]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[10]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[11]  Jorge Nocedal.  Conjugate Gradient Methods and Nonlinear Optimization , 1996 .

[12]  Elijah Polak,et al.  Optimization: Algorithms and Consistent Approximations , 1997 .

[13]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[14]  Ya-Xiang Yuan,et al.  Convergence Properties of Nonlinear Conjugate Gradient Methods , 1999, SIAM J. Optim..

[15]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[16]  Jiapu Zhang,et al.  Global Convergence of Conjugate Gradient Methods without Line Search , 2001, Ann. Oper. Res..

[17]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[18]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[19]  Ya-xiang,et al.  A NOTE ON THE NONLINEAR CONJUGATE GRADIENT METHOD , 2002 .

[20]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[21]  Huang Hai-dong.  A new conjugate gradient method for unconstrained optimization , 2007 .

[22]  G. Saridis,et al.  Journal of Optimization Theory and Applications Approximate Solutions to the Time-invariant Hamilton-jacobi-bellman Equation 1 , 1998 .

[23]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[24]  Jinhua Guo,et al.  A new family of conjugate gradient methods , 2009 .

[25]  Zengxin Wei,et al.  New line search methods for unconstrained optimization , 2009 .

[26]  N. Andrei.  Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization , 2009 .

[27]  J. Meza,et al.  Steepest descent , 2010 .

[28]  Yuhong Dai.  Nonlinear Conjugate Gradient Methods , 2011 .

[29]  Wah June Leong,et al.  A new class of nonlinear conjugate gradient coefficients with global convergence properties , 2012, Appl. Math. Comput..

[30]  I. Mohd,et al.  Solving unconstrained optimization with a new type of conjugate gradient method , 2014 .

[31]  M. Mamat,et al.  The global convergence properties of a conjugate gradient method , 2014 .

[32]  I. Mohd,et al.  The Convergence Properties of a New Type of Conjugate Gradient Methods , 2014 .

[33]  J. K. Liu,et al.  New hybrid conjugate gradient method for unconstrained optimization , 2014, Appl. Math. Comput..

[34]  I. Mohd,et al.  A new modification of Hestenes-Stiefel method with descent properties , 2014 .

[35]  I. Mohd,et al.  Global Convergence Properties of a New Class of Conjugate Gradient Method for Unconstrained Optimization , 2014 .

[36]  I. Mohd,et al.  The Proof of Sufficient Descent Condition for a New Type of Conjugate Gradient Methods , 2014 .

[37]  I. Mohd,et al.  A new conjugate gradient method for unconstrained optimization with sufficient descent , 2014 .

[38]  M. Mamat,et al.  A comparative study of two new conjugate gradient methods , 2015 .