In unconstrained minimization of a functions, the method of Davidon-Fletcher- Powell (a "variable-metric" method) enables the inverse of the Hessian H of f to be ap- proximated stepwise, using only values of the gradient of f. It is shown here that, by solv- ing a certain variational problem, formulas for the successive corrections to H can be de- rived which closely resemble Davidon's. A symmetric correction matrix is sought which minimizes a weighted Euclidean norm, and also satisfies the "DFP condition." Numerical tests are described, comparing the performance (on four "standard" test functions) of two variationally-derived formulas with Davidon's. A proof by Y. Bard, modelled on Fletcher and Powell's, showing that the new formulas give the exact H after N steps, is included in an appendix. 1. The DFP Method. The class of gradient methods for finding the uncon- strained minimum of a function f(x)* in which the direction Sk of the next iterative step from Xk to Xk+l is computed from a formula such as:
[1]
Bertram Klein.
Direct Use of Extremal Principles in Solving Certain Optimizing Problems Involving Inequalities
,
1955,
Oper. Res..
[2]
H. Chernoff,et al.
Gradient methods of maximization.
,
1955
.
[3]
Roger Fletcher,et al.
A Rapidly Convergent Descent Method for Minimization
,
1963,
Comput. J..
[4]
M. J. Box.
A Comparison of Several Current Optimization Methods, and the use of Transformations in Constrained Problems
,
1966,
Comput. J..
[5]
C. G. Broyden.
Quasi-Newton methods and their application to function minimisation
,
1967
.
[6]
J. Greenstadt.
On the relative efficiencies of gradient methods
,
1967
.
[7]
William C. Davidon,et al.
Variance Algorithm for Minimization
,
1968,
Comput. J..
[8]
D. Goldfarb.
A family of variable-metric methods derived by variational means
,
1970
.