Some numerical aspects of nonlinear estimation

Given a nonlinear estimation problem, it is well-known that one algorithm might produce a solution where another algorithm might fail. What is less well-known is that when both algorithms do produce solutions, one of the solutions might be more accurate than the other. In particular, one might give several accurate digits, while the other gives no accurate digits. Possible sources of the phenomenon are presented, including step-length, stopping rule, convergence criterion and method of derivative calculation. Additionally, practical advice for using nonlinear solvers is offered.

[1]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[2]  Philip E. Gill,et al.  Practical optimization , 1981 .

[3]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[4]  H. Vinod.  Review of GAUSS for Windows, including its numerical accuracy , 2000 .

[5]  David A. Belsley.  On the efficient computation of the nonlinear full-information maximum-likelihood estimator , 1980 .

[6]  Bruce D. McCullough,et al.  Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP , 1999 .

[7]  Richard E. Quandt,et al.  COMPUTATIONAL PROBLEMS AND METHODS , 1983 .

[8]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[9]  Bruce D. McCullough,et al.  Assessing the Reliability of Statistical Software: Part I , 1998 .

[10]  Paul Newbold,et al.  Adventures with ARIMA software , 1994 .

[11]  Bruce D. McCullough,et al.  The Numerical Reliability of Econometric Software , 1999 .

[12]  Jean-Marie Dufour,et al.  The Cochrane-Orcutt Procedure: Numerical Examples of Multiple Admissible Minima , 1980 .

[13]  K. Judd.  Numerical methods in economics , 1998 .

[14]  Charles G. Renfro,et al.  Benchmarks and software standards: A case study of GARCH procedures , 1998 .

[15]  J. MacKinnon,et al.  Estimation and inference in econometrics , 1994 .

[16]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[17]  William H. Press,et al.  Numerical Recipes: FORTRAN , 1988 .

[18]  John E. Dennis,et al.  An Adaptive Nonlinear Least-Squares Algorithm , 1977, TOMS.

[19]  M GayDavid,et al.  Algorithm 611: Subroutines for Unconstrained Minimization Using a Model/Trust-Region Approach , 1983 .

[20]  Charles G. Renfro.  Normative considerations in the development of a software package for econometric estimation , 1997 .

[21]  Robert B. Schnabel,et al.  Computational experience with confidence intervals for nonlinear least squares , 1986 .

[22]  David A. Heiser,et al.  On the accuracy of statistical procedures in Microsoft Excel 2007 , 1999, Comput. Stat. Data Anal..

[23]  R. Courant.  Differential and Integral Calculus , 1935 .

[24]  Marcel G. Dagenais,et al.  The Computation of FIML Estimates as Iterative Generalized Least Squares Estimates in Linear and Nonlinear Simultaneous Equations Models , 1978 .

[25]  Colin Rose,et al.  Mathematical Statistics with Mathematica , 2002 .

[26]  Christina Fraley.  Software Performance on Nonlinear Least-Squares Problems , 1989 .

[27]  Christoph W. Ueberhuber,et al.  Numerical Computation 2 , 1997 .