Bias reduction of maximum likelihood estimates

SUMMARY It is shown how, in regular parametric problems, the first-order term is removed from the asymptotic bias of maximum likelihood estimates by a suitable modification of the score function. In exponential families with canonical parameterization the effect is to penalize the likelihood by the Jeffreys invariant prior. In binomial logistic models, Poisson log linear models and certain other generalized linear models, the Jeffreys prior penalty function can be imposed in standard regression software using a scheme of iterative adjustments to the data.

[1]  H. Jeffreys.  An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[2]  J. Neyman,et al.  Consistent Estimates Based on Partially Consistent Observations , 1948 .

[3]  M. H. Quenouille.  Approximate Tests of Correlation in Time‐Series , 1949 .

[4]  B. Haldane.  THE ESTIMATION AND SIGNIFICANCE OF THE LOGARITHM OF A RATIO OF FREQUENCIES , 1956, Annals of human genetics.

[5]  F. J. Anscombe,et al.  On estimating binomial response relations , 1956 .

[6]  M. H. Quenouille.  NOTES ON BIAS IN ESTIMATION , 1956 .

[7]  B. L. Welch,et al.  On Formulae for Confidence Points Based on Integrals of Weighted Likelihoods , 1963 .

[8]  J. A. Hartigan,et al.  THE ASYMPTOTICALLY UNBIASED PRIOR DISTRIBUTION , 1965 .

[9]  D. Cox,et al.  A General Definition of Residuals , 1968 .

[10]  B. Efron.  Defining the Curvature of a Statistical Problem (with Applications to Second Order Efficiency) , 1975 .

[11]  D. Hinkley.  Conditional inference about a normal mean with known coefficient of variation , 1977 .

[12]  J. A. Anderson,et al.  Logistic Discrimination and Bias Correction in Maximum Likelihood Estimation , 1979 .

[13]  G. McLachlan.  A Note On Bias Correction in Maximum Likelihood Estimation with Logistic Discrimination , 1980 .

[14]  Norman E. Breslow,et al.  Odds ratio estimators when the data are sparse , 1981 .

[15]  O. Barndorff-Nielsen.  On a formula for the distribution of the maximum likelihood estimator , 1983 .

[16]  A. Albert,et al.  On the existence of maximum likelihood estimates in logistic regression models , 1984 .

[17]  R. Schaefer.  Bias correction in maximum likelihood logistic regression. , 1985, Statistics in medicine.

[18]  Donald B. Rubin,et al.  Logit-Based Interval Estimation for Binomial Data Using the Jeffreys Prior , 1987 .

[19]  D. Cox,et al.  Parameter Orthogonality and Approximate Conditional Inference , 1987 .

[20]  P. McCullagh.  Tensor Methods in Statistics , 1987 .

[21]  J. Copas.  Binary Regression Models for Contaminated Data , 1988 .

[22]  D. Cox,et al.  Analysis of Binary Data (2nd ed.). , 1990 .

[23]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[24]  Donald B. Rubin,et al.  Multiple Imputation of Industry and Occupation Codes in Census Public-use Samples Using Bayesian Logistic Regression , 1991 .

[25]  P. McCullagh,et al.  Bias Correction in Generalized Linear Models , 1991 .

[26]  David Firth,et al.  Bias reduction, the Jeffreys prior and GLIM , 1992 .

[27]  D. Firth.  Generalized Linear Models and Jeffreys Priors: An Iterative Weighted Least-Squares Approach , 1992 .

[28]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .