Some Investigations in Function Minimization

Significant advances have been made over the past decade in the development of powerful function minimization methods. Although the essential structure of each of these methods is fixed, several auxiliary features invariably remain to be selected by the user in any actual implementation. The effectiveness of the methods can, furthermore, be greatly influenced by the choice made by the user in this regard. The sensitivity of a variety of different methods to three of these features is examined through a series of computational experiments. These features are 1) the type of gradient information used (exact or approximated), 2) the precision requested in the solution of the line search subproblem, and 3) the superposition, on the basic algorithm, of a policy of periodic reinitialization.

[1]  Oren.  Self-scaling variable metric algorithms without line search for unconstrained minimization , 1973 .

[2]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[3]  Yonathan Bard,et al.  Comparison of Gradient Methods for the Solution of Nonlinear Parameter Estimation Problems , 1970 .

[4]  M. J. D. Powell,et al.  An Iterative Method for Finding Stationary Values of a Function of Several Variables , 1962, Comput. J..

[5]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[6]  S. Oren.  SELF-SCALING VARIABLE METRIC (SSVM) ALGORITHMS Part II: Implementation and Experiments*t , 1974 .

[7]  Bruce A. Murtagh,et al.  Computational Experience with Quadratically Convergent Minimisation Methods , 1970, Comput. J..

[8]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[9]  A. V. Levy,et al.  Numerical experiments on quadratically convergent algorithms for function minimization , 1970 .

[10]  M. J. Box.  A Comparison of Several Current Optimization Methods, and the use of Transformations in Constrained Problems , 1966, Comput. J..

[11]  K. Kawamura,et al.  On the rate of convergence of the conjugate gradient reset method with inaccurate linear minimizations , 1973 .

[12]  H. Sorenson.  Comparison of some conjugate direction procedures for function minimization , 1969 .

[13]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[14]  Willard I. Zangwill,et al.  Minimizing a function without calculating derivatives , 1967, Comput. J..

[15]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[16]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[17]  G. W. Stewart,et al.  A Modification of Davidon's Minimization Method to Accept Difference Approximations of Derivatives , 1967, JACM.

[18]  Louis G. Birta,et al.  The TEF/Davidon-Fletcher-Powell method in the computation of optimal controls , 1969 .