An adaptive three-term conjugate gradient method based on self-scaling memoryless BFGS matrix

Abstract Due to its simplicity and low memory requirement, conjugate gradient methods are widely used for solving large-scale unconstrained optimization problems. In this paper, we propose a three-term conjugate gradient method. The search direction is given by a symmetrical Perry matrix, which contains a positive parameter. The value of this parameter is determined by minimizing the distance of this matrix and the self-scaling memoryless BFGS matrix in the Frobenius norm. The sufficient descent property of the generated directions holds independent of line searches. The global convergence of the given method is established under Wolfe line search for general non-convex functions. Numerical experiments show that the proposed method is promising.

[1]  Reza Ghanbari,et al.  Two optimal Dai–Liao conjugate gradient methods , 2015 .

[2]  Hiroshi Yabe,et al.  Global Convergence Properties of Nonlinear Conjugate Gradient Methods with Modified Secant Condition , 2004, Comput. Optim. Appl..

[3]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[4]  Reza Ghanbari,et al.  The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices , 2014, Eur. J. Oper. Res..

[5]  Boris Polyak.  Some methods of speeding up the convergence of iteration methods , 1964 .

[6]  Neculai Andrei,et al.  A simple three-term conjugate gradient algorithm for unconstrained optimization , 2013, J. Comput. Appl. Math..

[7]  Neculai Andrei.  A new three-term conjugate gradient algorithm for unconstrained optimization , 2014, Numerical Algorithms.

[8]  M. Fukushima,et al.  A modified BFGS method and its global convergence in nonconvex minimization , 2001 .

[9]  Emmanuel J. Candès,et al.  Adaptive Restart for Accelerated Gradient Schemes , 2012, Foundations of Computational Mathematics.

[10]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[11]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[12]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[13]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[14]  Jianzhon Zhang,et al.  Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations , 2001 .

[15]  Yu-Hong Dai,et al.  A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search , 2013, SIAM J. Optim..

[16]  Masoud Fatemi.  An Optimal Parameter for Dai–Liao Family of Conjugate Gradient Methods , 2016, J. Optim. Theory Appl..

[17]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[18]  Clóvis C. Gonzaga,et al.  Fine tuning Nesterov’s steepest descent algorithm for differentiable convex programming , 2012, Mathematical Programming.

[19]  C. Kou,et al.  An improved nonlinear conjugate gradient method with an optimal property , 2014 .

[20]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .

[21]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[22]  Li Zhang,et al.  A nonlinear conjugate gradient method based on the MBFGS secant condition , 2006, Optim. Methods Softw..

[23]  Yurii Nesterov,et al.  Subgradient methods for huge-scale optimization problems , 2013, Mathematical Programming.

[24]  Guoyin Li,et al.  New quasi-Newton methods for unconstrained optimization problems , 2006, Appl. Math. Comput..

[25]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[26]  Avinoam Perry,et al.  Technical Note - A Modified Conjugate Gradient Algorithm , 1978, Oper. Res..

[27]  Neculai Andrei,et al.  On three-term conjugate gradient algorithms for unconstrained optimization , 2013, Appl. Math. Comput..

[28]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[29]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[30]  Guoyin Li,et al.  New conjugacy condition and related new conjugate gradient methods for unconstrained optimization , 2007 .

[31]  Neculai Andrei,et al.  Accelerated adaptive Perry conjugate gradient algorithms based on the self-scaling memoryless BFGS update , 2017, J. Comput. Appl. Math..

[32]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[33]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[34]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[35]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[36]  Reza Ghanbari,et al.  A descent family of Dai–Liao conjugate gradient methods , 2014, Optim. Methods Softw..

[37]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..