A modified BFGS algorithm based on a hybrid secant equation

By making a convex combination of the modified secant equations proposed by Yuan and Wei et al., a hybrid secant equation and also, a modified BFGS algorithm is proposed. The hybridization parameter is effectively computed using the available information of recent iterations. Under proper conditions, it is shown that the proposed algorithm is globally, locally and superlinearly convergent. By using the performance profile introduced by Dolan and Moré, a comparison between the implementations of the proposed algorithm and two efficient modified BFGS algorithms proposed by Yuan and Wei et al., on a set of unconstrained optimization test problems from the CUTEr collection, is done. Numerical results demonstrating the efficiency of the proposed modified BFGS algorithm are reported.

[1]  H. Y. Huang.  Unified approach to quadratically convergent algorithms for function minimization , 1970 .

[2]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[3]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[4]  Gonglin Yuan,et al.  The Superlinear Convergence of a Modified BFGS-Type Method for Unconstrained Optimization , 2004, Comput. Optim. Appl..

[5]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .

[6]  L. H. Chen,et al.  A Modified Quasi-Newton Method for Structured Optimization with Partial Information on the Hessian , 2006, Comput. Optim. Appl..

[7]  Jianzhon Zhang,et al.  Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations , 2001 .

[8]  John E. Dennis,et al.  On the Local and Superlinear Convergence of Quasi-Newton Methods , 1973 .

[9]  Issam A. R. Moghrabi,et al.  Multi-step quasi-Newton methods for optimization , 1994 .

[10]  Hiroshi Yabe,et al.  Global Convergence Properties of Nonlinear Conjugate Gradient Methods with Modified Secant Condition , 2004, Comput. Optim. Appl..

[11]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[12]  Jingfeng Zhang,et al.  New Quasi-Newton Equation and Related Methods for Unconstrained Optimization , 1999 .

[13]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[14]  H. Yabe,et al.  Local and superlinear convergence of quasi-Newton methods based on modified secant conditions , 2007 .

[15]  Guoyin Li,et al.  New quasi-Newton methods for unconstrained optimization problems , 2006, Appl. Math. Comput..

[16]  W. Davidon.  Conic Approximations and Collinear Scalings for Optimizers , 1980 .

[17]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[18]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[19]  J. Dennis,et al.  Convergence theory for the structured BFGS secant method with an application to nonlinear least squares , 1989 .

[20]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[21]  Reza Ghanbari,et al.  Two new conjugate gradient methods based on modified secant equations , 2010, J. Comput. Appl. Math..

[22]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[23]  Issam A. R. Moghrabi,et al.  Using function-values in multi-step quasi-Newton methods , 1996 .

[24]  Jianguo Liu,et al.  A modified BFGS method and its superlinear convergence in nonconvex minimization with general line search rule , 2008 .

[25]  Ya-Xiang Yuan,et al.  A modified BFGS algorithm for unconstrained optimization , 1991 .

[26]  M. Fukushima,et al.  A modified BFGS method and its global convergence in nonconvex minimization , 2001 .

[27]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[28]  Keyvan Amini,et al.  A new structured quasi-Newton algorithm using partial information on Hessian , 2010, J. Comput. Appl. Math..