Relative Sizes of the Hessian Terms in Nonlinear Parameter Estimation

The computational estimation of parameters in nonlinear models leads to nonlinear least squares problems with Hessians having a particular structure: $H = J^{\text{T}} J + B$, with J the Jacobian matrix and B arising from second derivative terms. In practice, it has been noticed that B is very often considerably smaller than $J^{\text{T}} J$, particularly near a local minimum, so that H can often be well approximated by the first derivative term. In this paper an attempt is made to give some explanation for this behaviour and to indicate how to check for it during the computational process.