A new method for evaluating the log-likelihood gradient, the Hessian, and the Fisher information matrix for linear dynamic systems

A method is presented for evaluating the log-likelihood gradient (score), the Hessian, and the Fisher information matrix of the parameters of linear dynamic stochastic systems. The method incorporates the optimal Kalman smoothing equations and is therefore ideal for simultaneous state estimation and parameter identification. The result can be used for efficient implementation of gradient-based algorithms for maximum-likelihood identification of the unknown system parameters and for assessing the mean-square estimation accuracy. >

[1]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[2]  Raman K. Mehra,et al.  Frequency-domain synthesis of optimal inputs for multiinput-multioutput (MIMO) systems with process noise , 1974, CDC 1974.

[3]  Karl Johan Åström.  Maximum Likelihood and Prediction Error Methods , 1981 .

[4]  Thomas Kailath,et al.  Supplement to "a survey of data smoothing" , 1975, Autom..

[5]  R. Mehra.  Synthesis of Optimal Inputs for Multiinput-Multioutput (Mimo) Systems with Process Noise Part I: Frequenc Y-Domain Synthesis Part II: Time-Domain Synthesis , 1976 .

[6]  David Hinkley,et al.  Theory of Statistical Estimation: The 1925 Paper , 1980 .

[7]  Uri Shaked,et al.  Explicit solutions to the singular discrete finite-time linear estimation problem , 1986 .

[8]  I. Meilijson.  A fast improvement to the EM algorithm on its own terms , 1989 .

[9]  Uri Shaked,et al.  Explicit solutions to the linear optimal estimation problem of discrete time invariant linear processes , 1982 .

[10]  J. Meditch.  A survey of data smoothing for linear and nonlinear dynamic systems , 1973 .

[11]  Thomas Kailath,et al.  Lattice filter parameterization and modeling of nonstationary processes , 1984, IEEE Trans. Inf. Theory.

[12]  H. Weinert,et al.  Discrete-time complementary models and smoothing algorithms: The correlated noise case , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[13]  Salvatore D. Morgera,et al.  The Levinson recurrence and fast algorithms for solving Toeplitz systems of linear equations , 1987, IEEE Trans. Acoust. Speech Signal Process..

[14]  Lennart Ljung,et al.  A unified approach to smoothing formulas , 1976, Autom..

[15]  M. Segal,et al.  A new method for evaluating the log-likelihood gradient (score) of linear dynamic systems , 1988 .

[16]  R. Mehra,et al.  Computational aspects of maximum likelihood estimation and reduction in sensitivity function calculations , 1974 .

[17]  N. Sandell,et al.  MAXIMUM LIKELIHOOD IDENTIFICATION OF STATE SPACE MODELS FOR LINEAR DYNAMIC SYSTEMS , 1978 .

[18]  Uri Shaked,et al.  Mixed mode solution to the partially singular discrete-time filtering problem by sequential decomposition , 1988 .

[19]  R. Shumway,et al.  AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[20]  S. Hill.  Reduced gradient computation in prediction error identification , 1985 .

[21]  A. Willsky,et al.  On the fixed-interval smoothing problem † , 1981 .