EFFECT OF ALGORITHM PARAMETERS ON NUMERICAL PERFORMANCE OF VARIABLE METRIC ACCEPTABLE POINT ALGORITHMS (AS REPRESENTED BY BFGS-ARMIJO)

Variable metric acceptable point algorithms generally contain several user-specified tuning parameters, the selection of which can greatly effect overall convergence. BFGS-Armijo is a practical, easily-coded, generally effective, and widely-used algorithm and is reasonably representative of this class of algorithm. Using three widely-used test functions and three “more difficult” test functions, the influence of the three Armijo parameters are investigated in terms of (i) literature-suggested values, (ii) optimal values of these parameters, and (iii) performance in ranges as suggested by (ii). Extensive numerical results are reported for algorithm effectiveness (equivalent function evaluations, multivariate iterations) and sensitivity. It is found that the literature-suggested values of the parameters are reasonable choices, but are not necessarily optimal. Indeed, it seems that there are optimal ranges of these parameters which include most of the suggested values but which are somewhat broader than is i...

[1]  L. Nazareth.  A conjugate direction algorithm without line searches , 1977 .

[2]  Elijah Polak,et al.  Computational methods in optimization , 1971 .

[3]  M. C. Biggs.  Minimization Algorithms Making Use of Non-quadratic Properties of the Objective Function , 1971 .

[4]  J. F. Price,et al.  An effective algorithm for minimization , 1967 .

[5]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[6]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[7]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[8]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[9]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[10]  M. A. Townsend,et al.  A generalized direct search acceptable-point technique for use with descent-type multivariate algorithms , 1980 .

[11]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[12]  D. Shanno,et al.  Numerical comparison of several variable-metric algorithms , 1978 .

[13]  L. C. W. Dixon.  Conjugate Directions without Linear Searches , 1973 .

[14]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[15]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .