The Hager–Zhang conjugate gradient algorithm for large-scale nonlinear equations

ABSTRACT In this paper, the Hager–Zhang (HZ) conjugate gradient (CG) algorithm is studied for large-scale smooth optimization problems. (i) Some results of the HZ CG method for smooth unconstrained optimization problems are given, and a modified HZ (MHZ) CG method is proposed; (ii) the HZ and MHZ CG methods for nonlinear equations are analysed, the global convergence is established and numerical results for large-scale nonlinear equation problems are reported (1,00,000 variables).

[1]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[2]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[3]  Dong-Hui Li,et al.  n-step quadratic convergence of the MPRP method with a restart strategy , 2011, J. Comput. Appl. Math..

[4]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[5]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[6]  Natasa Krejic,et al.  Practical Quasi-Newton algorithms for singular nonlinear systems , 2010, Numerical Algorithms.

[7]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[8]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[9]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..

[10]  C. Kanzow.  Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties , 2004 .

[11]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .

[12]  Xiwen Lu,et al.  A conjugate gradient method with descent direction for unconstrained optimization , 2009, J. Comput. Appl. Math..

[13]  W. Burmeister.  Die Konvergenzordnung des Fletcher‐Powell‐Algorithmus , 1973 .

[14]  Gonglin Yuan,et al.  Limited memory BFGS method with backtracking for symmetric nonlinear equations , 2011, Math. Comput. Model..

[15]  Bobby Schnabel,et al.  Tensor methods for large sparse systems of nonlinear equations , 1996, Math. Program..

[16]  Qingna Li,et al.  A class of derivative-free methods for large-scale nonlinear monotone equations , 2011 .

[17]  Xiwen Lu,et al.  A BFGS trust-region method for nonlinear equations , 2011, Computing.

[18]  Liying Liu,et al.  The convergence properties of some new conjugate gradient methods , 2006, Appl. Math. Comput..

[19]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[20]  William W. Hager,et al.  Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent , 2006, TOMS.

[21]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[22]  Xiwen Lu,et al.  BFGS trust-region method for symmetric nonlinear equations , 2009 .

[23]  Zhifang,et al.  CHARACTERISTICS OF INVESTORS' RISK PREFERENCE FOR STOCK MARKETS , 2014 .

[24]  Marcos Raydan,et al.  Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems , 2003, Optim. Methods Softw..

[25]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[26]  Yuhong Dai.  Nonlinear Conjugate Gradient Methods , 2011 .

[27]  Dong-Hui Li,et al.  A modified Fletcher-Reeves-Type derivative-free method for symmetric nonlinearequations , 2011 .

[28]  Ph. L. Toint.  Numerical solution of large sets of algebraic nonlinear equations , 1986 .

[29]  Gonglin Yuan,et al.  Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems , 2009, Optim. Lett..

[30]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[31]  Dong-Hui Li,et al.  Descent Directions of Quasi-Newton Methods for Symmetric Nonlinear Equations , 2002, SIAM J. Numer. Anal..

[32]  Liqun Qi,et al.  On the Convergence of a Trust-Region Method for Solving Constrained Nonlinear Equations with Degenerate Solutions , 2004 .

[33]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[34]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[35]  Ya-Xiang Yuan,et al.  Subspace methods for large scale nonlinear equations and nonlinear least squares , 2009 .

[36]  Mikhail V. Solodov,et al.  A Globally Convergent Inexact Newton Method for Systems of Monotone Equations , 1998 .

[37]  K. Ritter.  On the rate of superlinear convergence of a class of variable metric methods , 1980 .

[38]  A. I. Cohen.  Rate of convergence of several conjugate gradient algorithms. , 1972 .

[39]  Gonglin Yuan,et al.  A BFGS algorithm for solving symmetric nonlinear equations , 2013 .

[40]  Xiwen Lu,et al.  A new backtracking inexact BFGS method for symmetric nonlinear equations , 2008, Comput. Math. Appl..

[41]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[42]  Ya-Xiang Yuan.  Analysis on the conjugate gradient method , 1993 .

[43]  Wufan Chen,et al.  Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization , 2008, Optim. Methods Softw..

[44]  Gonglin Yuan,et al.  A Modified Hestenes-Stiefel Conjugate Gradient Algorithm for Large-Scale Optimization , 2013 .

[45]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[46]  Zhen-Jun Shi,et al.  Convergence of line search methods for unconstrained optimization , 2004, Appl. Math. Comput..

[47]  Xiaohong Chen,et al.  A modified Perry's conjugate gradient method-based derivative-free method for solving large-scale nonlinear monotone equations , 2015, Appl. Math. Comput..

[48]  Weijun Zhou,et al.  Spectral gradient projection method for solving nonlinear monotone equations , 2006 .

[49]  Guoyin Li,et al.  Global convergence of the Polak-Ribière-Polyak conjugate gradient method with an Armijo-type inexact line search for nonconvex unconstrained optimization problems , 2008, Math. Comput..

[50]  C. Storey,et al.  Global convergence result for conjugate gradient methods , 1991 .

[51]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Gauss-Newton-Based BFGS Method for Symmetric Nonlinear Equations , 1999, SIAM J. Numer. Anal..

[52]  Boris Polyak.  The conjugate gradient method in extreme problems , 2015 .

[53]  Xiwen Lu,et al.  A modified PRP conjugate gradient method , 2009, Ann. Oper. Res..

[54]  Wanyou Cheng,et al.  A PRP type method for systems of monotone equations , 2009, Math. Comput. Model..

[55]  Chuanwei Wang,et al.  A projection method for a system of nonlinear monotone equations with convex constraints , 2007, Math. Methods Oper. Res..

[56]  Mengnien Wu,et al.  Numerical Schubert Calculus by the Pieri Homotopy Algorithm , 2002, SIAM J. Numer. Anal..

[57]  Gonglin Yuan,et al.  A modified Polak–Ribière–Polyak conjugate gradient algorithm for large-scale optimization problems , 2014 .

[58]  Yong Wang,et al.  A new trust region method for nonlinear equations , 2003, Math. Methods Oper. Res..

[59]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[60]  Luigi Grippo,et al.  Nonmonotone derivative-free methods for nonlinear equations , 2007, Comput. Optim. Appl..

[61]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[62]  D. Touati-Ahmed,et al.  Efficient hybrid conjugate gradient techniques , 1990 .

[63]  Marco Sciandrone,et al.  A Truncated Nonmonotone Gauss-Newton Method for Large-Scale Nonlinear Least-Squares Problems , 2006, Comput. Optim. Appl..

[64]  Yunhai Xiao,et al.  Spectral gradient projection method for monotone nonlinear equations with convex constraints , 2009 .

[65]  Maojun Zhang,et al.  A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations , 2015, J. Comput. Appl. Math..

[66]  Neculai Andrei,et al.  Another hybrid conjugate gradient algorithm for unconstrained optimization , 2008, Numerical Algorithms.