Analysis of data-based methods for approximating fisher information in the scalar case

The Fisher information matrix (FIM) has long been of interest in applied mathematics and statistics for its various uses. Its scalar case, the Fisher information number (FIN), is also widely used. However, in many cases it is challenging to obtain the true value for the Fisher information so we need to use approximations. In this paper, we compare the accuracy of two major approximation methods for FIN and provide numerical results as illustrations to our analysis.

[1]  Y. Favennec.  Hessian and Fisher Matrices For Error Analysis in Inverse Heat Conduction Problems , 2007 .

[2]  J. Spall.  Monte Carlo Computation of the Fisher Information Matrix in Nonstandard Settings , 2005 .

[3]  James C. Spall,et al.  Relative performance of expected and observed fisher information in covariance estimation for maximum likelihood estimates , 2012, 2012 American Control Conference (ACC).

[4]  Shenghan Guo,et al.  Comparison of Accuracy for Methods to Approximate Fisher Information in the Scalar Case , 2014, ArXiv.

[5]  James C. Spall,et al.  Preliminary results on relative performance of expected and observed fisher information , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[6]  James C. Spall,et al.  Identification for Systems With Binary Subsystems , 2014, IEEE Transactions on Automatic Control.

[7]  B. Morgan,et al.  Negative Score Test Statistic , 2007 .

[8]  James C. Spall,et al.  Parameter identification for state-space models with nuisance parameters , 1990 .

[9]  J. L. Maryak,et al.  A Feasible Bayesian Estimator of Quantiles for Projectile Accuracy From Non-iid Data , 1992 .

[10]  W. Rudin.  Principles of mathematical analysis , 1964 .

[11]  B. Efron,et al.  Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information , 1978 .

[12]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[13]  S. D. Hill,et al.  Least-informative Bayesian prior distributions for finite samples based on information theory , 1990 .