Computational aspects of maximum likelihood estimation and reduction in sensitivity function calculations

This paper discusses numerical aspects of computing maximum likelihood estimates for linear dynamical systems in state-vector form. Different gradient-based nonlinear programming methods are discussed in a unified framework and their applicability to maximum likelihood estimation is examined. The problems due to singular Hessian or singular information matrix that are common in practice are discussed in detail and methods for their solution are proposed. New results on the calculation of state sensitivity functions via reduced order models are given. Several methods for speeding convergence and reducing computation time are also discussed.

[1]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[2]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[3]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[4]  Karl Johan Åström,et al.  Numerical Identification of Linear Dynamic Systems from Normal Operating Records , 1965 .

[5]  Adi Ben-Israel.  A Newton-Raphson method for the solution of systems of equations , 1966 .

[6]  George A. Barnard,et al.  The use of the likelihood function in statistical practice , 1967 .

[7]  S. Mcreynolds.  The successive sweep method and dynamic programming , 1967 .

[8]  J. Greenstadt.  On the relative efficiencies of gradient methods , 1967 .

[9]  R. Jennrich,et al.  Application of Stepwise Regression to Non-Linear Estimation , 1968 .

[10]  W. Perkins,et al.  Generation of sensitivity functions for linear systems using low-order models , 1969 .

[11]  Kenneth Steiglitz,et al.  ESTIMATION OF DISTRIBUTED LAGS , 1970 .

[12]  Yonathan Bard,et al.  Comparison of Gradient Methods for the Solution of Nonlinear Parameter Estimation Problems , 1970 .

[13]  Raman K. Mehra,et al.  Maximum likelihood identification of aircraft parameters , 1970 .

[14]  T. Bohlin,et al.  On the maximum likelihood method of identification , 1970 .

[15]  M. J. D. Powell,et al.  Recent advances in unconstrained optimization , 1971, Math. Program..

[16]  Dallas G. Denery.  Simplification in the computation of the sensitivity functions for constant coefficient linear systems , 1971 .

[17]  Arun K. Sood,et al.  Sensitivity functions for multi-input linear time-invariant systems—II. Minimal-order models† , 1972 .

[18]  E. Polak,et al.  An adaptive precision gradient method for optimal control. , 1973 .

[19]  H. Akaike.  Maximum likelihood identification of Gaussian autoregressive moving average models , 1973 .

[20]  H. Akaike.  Block Toeplitz Matrix Inversion , 1973 .

[21]  M. Morf,et al.  Square-root algorithms for least-squares estimation , 1975 .