On damping parameters of Levenberg-Marquardt algorithm for nonlinear least square problems

The Levenberg-Marquardt (LM) algorithm is a widely used method for solving problems related to nonlinear least squares. The method depends on a nonlinear parameter μ known as self-scaling parameter that affects the performance of the algorithm. In this paper we examine the effect of various choice of parameters and of relaxing the line search. Numerical results obtained are used to compare the performance using standard test problems which show that the proposed alternatives are promising.

[1]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[2]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[3]  Li Min Tang,et al.  A Regularization Homotopy Iterative Method for Ill-Posed Nonlinear Least Squares Problem and its Application , 2011 .

[4]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[5]  M. J. D. Powell,et al.  An Iterative Method for Finding Stationary Values of a Function of Several Variables , 1962, Comput. J..

[6]  M. A. Mohamed,et al.  SOLVING FUZZY NONLINEAR EQUATION VIA LEVENBERG-MARQUARDT METHOD , 2018 .

[7]  C. Fraley,et al.  Algorithms for Nonlinear Least-Squares Problems , 1988 .

[8]  Kaj Madsen,et al.  Methods for Non-Linear Least Squares Problems , 1999 .

[9]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[10]  Hassan Mohammad,et al.  A brief survey of methods for solving nonlinear least-squares problems , 2019, Numerical Algebra, Control & Optimization.

[11]  M. J. Box.  A Comparison of Several Current Optimization Methods, and the use of Transformations in Constrained Problems , 1966, Comput. J..

[12]  Serge Gratton,et al.  Approximate Gauss-Newton Methods for Nonlinear Least Squares Problems , 2007, SIAM J. Optim..

[13]  Feng Ding,et al.  Maximum likelihood least squares identification method for input nonlinear finite impulse response moving average systems , 2012, Math. Comput. Model..

[14]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[15]  R. Jennrich,et al.  Application of Stepwise Regression to Non-Linear Estimation , 1968 .

[16]  M. D. Hebden,et al.  An algorithm for minimization using exact second derivatives , 1973 .

[17]  Ferdinand Freudenstein,et al.  Numerical Solution of Systems of Nonlinear Equations , 1963, JACM.

[18]  J. Dennis,et al.  Derivative free analogues of the Levenberg-Marquardt and Gauss algorithms for nonlinear least squares approximation , 1971 .

[19]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[20]  Anastasia Cornelio,et al.  Regularized nonlinear least squares methods for hit position reconstruction in small gamma cameras , 2011, Appl. Math. Comput..

[21]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[22]  G. Golub,et al.  Separable nonlinear least squares: the variable projection method and its applications , 2003 .

[23]  Günter Wozny,et al.  Nonlinear ill-posed problem analysis in model-based parameter estimation and experimental design , 2015, Comput. Chem. Eng..

[24]  Stefan Henn.  A Levenberg–Marquardt Scheme for Nonlinear Image Registration , 2003 .

[25]  M. R. Osborne.  Nonlinear least squares — the Levenberg algorithm revisited , 1976, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[26]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..