The information matrix in control: computation and some applications

The Fisher information matrix plays a central role in estimation and input design for input-output systems. This matrix provides a summary of the amount of information in the data relative to the quantities of interest. Some of the specific applications of the information matrix include confidence region calculation for parameter estimates, the determination of optimal inputs for model building, the providing of a bound on the best possible performance in an adaptive system (such as a control system), and producing uncertainty bounds on predictions (such as with neural network). However, the analytical calculation of the information matrix is often a difficult or impossible task. This is especially the case with nonlinear models such as neural networks. This paper briefly reviews some of the applications of the information matrix in control and describes a resampling-based method for computing the information matrix. This method applies in problems of arbitrary difficulty and is relatively easy to implement.

[1]  D. M. Titterington,et al.  Recent advances in nonlinear experiment design , 1989 .

[2]  James C. Spall,et al.  A one-measurement form of simultaneous perturbation stochastic approximation , 1997, Autom..

[3]  P. Chaudhuri,et al.  On efficient designing of nonlinear experiments , 1995 .

[4]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[5]  J. T. Hwang,et al.  Prediction Intervals for Artificial Neural Networks , 1997 .

[6]  Calyampudi R. Rao,et al.  Linear statistical inference and its applications , 1965 .

[7]  J. Spall.  Resampling-based calculation of the information matrix for general identification problems , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[8]  B. Lindsay,et al.  On second-order optimality of the observed Fisher information , 1997 .

[9]  B. Hoadley.  Asymptotic Properties of Maximum Likelihood Estimators for the Independent Not Identically Distributed Case , 1971 .

[10]  P. Hall.  The Bootstrap and Edgeworth Expansion , 1992 .

[11]  Robert Tibshirani,et al.  Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[12]  H. White.  Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .

[13]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[14]  H. Kushner,et al.  Stochastic approximation with averaging and feedback: rapidly convergent "on-line" algorithms , 1995, IEEE Trans. Autom. Control..

[15]  George Chryssolouris,et al.  Confidence interval prediction for neural network models , 1996, IEEE Trans. Neural Networks.