Gauss–Newton method

The Gauss-Newton method for nonlinear regression is described, along with its close relationship with the Fisher scoring method. Its many extensions, sometimes in the form of Fisher scoring, are briefly reviewed, including those for coping with large-residual problems in nonlinear regression, for optimizing a likelihood or other performance criterion, and for dealing with parameters with constraints. WIREs Comput Stat 2012 doi: 10.1002/wics.1202

[1]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[2]  John E. Dennis,et al.  An Adaptive Nonlinear Least-Squares Algorithm , 1977, TOMS.

[3]  Yong Wang.  On fast computation of the non‐parametric maximum likelihood estimate of a mixing distribution , 2007 .

[4]  G. Seber,et al.  Nonlinear Regression: Seber/Nonlinear Regression , 2005 .

[5]  Rory A. Fisher,et al.  Theory of Statistical Estimation , 1925, Mathematical Proceedings of the Cambridge Philosophical Society.

[6]  C. Gauss,et al.  Theory of the motion of the heavenly bodies moving about the sun in conic sections : a translation of Theoria motus. , 1964 .

[7]  P. Green.  Iteratively reweighted least squares for maximum likelihood estimation , 1984 .

[8]  Yong Wang,et al.  Maximum likelihood computation based on the Fisher scoring and Gauss-Newton quadratic approximations , 2007, Comput. Stat. Data Anal..

[9]  Yong Wang.  Minimum disparity computation via the iteratively reweighted least integrated squares algorithms , 2007, Comput. Stat. Data Anal..

[10]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[11]  Yong Wang,et al.  Fisher scoring: An interpolation family and its Monte Carlo implementations , 2010, Comput. Stat. Data Anal..

[12]  B. Lindsay.  Efficiency versus robustness : the case for minimum Hellinger distance and related methods , 1994 .

[13]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[14]  J. Betts.  Solving the nonlinear least square problem: Application of a general method , 1976 .

[15]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[16]  R. W. Wedderburn.  Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method , 1974 .