An adaptive scaled BFGS method for unconstrained optimization

A new adaptive scaled Broyden-Fletcher-Goldfarb-Shanno (BFGS) method for unconstrained optimization is presented. The third term in the standard BFGS update formula is scaled in order to reduce the large eigenvalues of the approximation to the Hessian of the minimizing function. Under the inexact Wolfe line search conditions, the global convergence of the adaptive scaled BFGS method is proved in very general conditions without assuming the convexity of the minimizing function. Using 80 unconstrained optimization test functions with a medium number of variables, the preliminary numerical experiments show that this variant of the scaled BFGS method is more efficient than the standard BFGS update or than some other scaled BFGS methods.

[1]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[2]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[3]  M. J. D. Powell,et al.  How bad are the BFGS and DFP methods when the objective function is quadratic? , 1986, Math. Program..

[4]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[5]  Andreas Griewank,et al.  The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients , 1991, Math. Program..

[6]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[7]  Dong-Hui Li,et al.  Spectral Scaling BFGS Method , 2010 .

[8]  R. Kellogg.  A nonlinear alternating direction method , 1969 .

[9]  Aiping Liao,et al.  Modifying the BFGS method , 1997, Oper. Res. Lett..

[10]  M. C. Biggs.  Minimization Algorithms Making Use of Non-quadratic Properties of the Objective Function , 1971 .

[11]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[12]  Jorge Nocedal,et al.  Analysis of a self-scaling quasi-Newton method , 1993, Math. Program..

[13]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[14]  Ya-Xiang Yuan,et al.  Optimization theory and methods , 2006 .

[15]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[16]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[17]  Siam Rfview,et al.  CONVERGENCE CONDITIONS FOR ASCENT METHODS , 2016 .

[18]  Philip E. Gill,et al.  Reduced-Hessian Quasi-Newton Methods for Unconstrained Optimization , 2001, SIAM J. Optim..

[19]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[20]  M. Al-Baali.  Numerical Experience with a Class of Self-Scaling Quasi-Newton Algorithms , 1998 .

[21]  M. Fukushima,et al.  A modified BFGS method and its global convergence in nonconvex minimization , 2001 .

[22]  Mehiddin Al-Baali.  Global and Superlinear Convergence of a Restricted Class of Self-Scaling Methods with Inexact Line Searches, for Convex Functions , 1998, Comput. Optim. Appl..

[23]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[24]  L. Dixon.  Variable metric algorithms: Necessary and sufficient conditions for identical behavior of nonquadratic functions , 1972 .

[25]  M. J. D. Powell,et al.  Updating conjugate directions by the BFGS formula , 1987, Math. Program..

[26]  E. Spedicato.  Algorithms for continuous optimization : the state of the art , 1994 .

[27]  Yu-Hong Dai,et al.  Convergence Properties of the BFGS Algoritm , 2002, SIAM J. Optim..

[28]  Jorge Nocedal,et al.  On the Behavior of Broyden's Class of Quasi-Newton Methods , 1992, SIAM J. Optim..

[29]  Walter F. Mascarenhas,et al.  The BFGS method with exact line searches fails for non-convex objective functions , 2004, Math. Program..

[30]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[31]  M. C. Biggs.  A Note on Minimization Algorithms which make Use of Non-quardratic Properties of the Objective Function , 1973 .

[32]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[33]  D. Luenberger,et al.  SELF-SCALING VARIABLE METRIC ( SSVM ) ALGORITHMS Part I : Criteria and Sufficient Conditions for Scaling a Class of Algorithms * t , 2007 .

[34]  M. Powell.  On the Convergence of the Variable Metric Algorithm , 1971 .

[35]  Roger Fletcher,et al.  An Overview of Unconstrained Optimization , 1994 .

[36]  Jorge Nocedal,et al.  Theory of algorithms for unconstrained optimization , 1992, Acta Numerica.

[37]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[38]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[39]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[40]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[41]  Jianzhon Zhang,et al.  Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations , 2001 .

[42]  Ya-Xiang Yuan,et al.  A modified BFGS algorithm for unconstrained optimization , 1991 .

[43]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .