Derivative free analogues of the Levenberg-Marquardt and Gauss algorithms for nonlinear least squares approximation

In this paper we give two derivative-free computational algorithms for nonlinear least squares approximation. The algorithms are finite difference analogues of the Levenberg-Marquardt and Gauss methods. Local convergence theorems for the algorithms are proven. In the special case when the residuals are zero at the minimum, we show that certain computationally simple choices of the parameters lead to quadratic convergence. Numerical examples are included.

[1]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[2]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[3]  G. R. Hext,et al.  Sequential Application of Simplex Designs in Optimisation and Evolutionary Operation , 1962 .

[4]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[5]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[6]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[7]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[8]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[9]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[10]  John Grover Barnes,et al.  A KDF9 ALGOL list-processing scheme , 1965, Comput. J..

[11]  M. J. D. Powell,et al.  A Method for Minimizing a Sum of Squares of Non-Linear Functions Without Calculating Derivatives , 1965, Comput. J..

[12]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[13]  John Grover Barnes,et al.  An Algorithm for Solving Non-Linear Equations Based on the Secant Method , 1965, Comput. J..

[14]  M. J. Box.  A Comparison of Several Current Optimization Methods, and the use of Transformations in Constrained Problems , 1966, Comput. J..

[15]  J. Greenstadt.  Variations on variable-metric methods. (With discussion) , 1970 .

[16]  Yonathan Bard,et al.  Comparison of Gradient Methods for the Solution of Nonlinear Parameter Estimation Problems , 1970 .

[17]  J. Greenstadt.  Variations on Variable-Metric Methods , 1970 .

[18]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.