A comparative study of the bias corrected estimates in logistic regression

Logistic regression is frequently used in many areas of applied statistics. The maximum likelihood estimates (MLE) of the logistic regression parameters are usually computed using the iterative Newton—Raphson method. It is well known that these estimates are biased. Several methods are proposed to correct the bias of these estimates. Among them Firth (1993) and Cordeiro and McCullagh (1991) proposed two promising methods. The conditional exact method (CMLE) is popular for small-sample estimates, and is available in many software packages. In this article we compare these methods in terms of their bias. In general, our extensive simulations show that the methods proposed by Cordeiro and McCullagh and by Firth work well, though Cordeiro and McCullagh is slightly better in our simulations. In case of separation, Firth or CMLE can be used; however, a judicious approach is required when there is a wide variation in results. Two real data analyses are given exhibiting these properties. The data analysis also includes bootstrap results.

[1]  Nitin R. Patel,et al.  Exact logistic regression: theory and examples. , 1995, Statistics in medicine.

[2]  Georg Heinze,et al.  A comparative investigation of methods for logistic regression with separated or nearly separated data , 2006, Statistics in medicine.

[3]  Nitin R. Patel,et al.  Computing Distributions for Exact Logistic Regression , 1987 .

[4]  M. Schemper,et al.  A solution to the problem of separation in logistic regression , 2002, Statistics in medicine.

[5]  Stanley Lemeshow,et al.  Applied Logistic Regression, Second Edition , 1989 .

[6]  D. Spitzner.  Risk‐reducing shrinkage estimation for generalized linear models , 2005 .

[7]  W. Parks,et al.  A hospital-based prospective study of perinatal infection with human immunodeficiency virus type 1. , 1991, The Journal of pediatrics.

[8]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[9]  D. Firth.  Bias reduction of maximum likelihood estimates , 1993 .

[10]  Georg Heinze,et al.  Fixing the nonconvergence bug in logistic regression with SPLUS and SAS , 2003, Comput. Methods Programs Biomed..

[11]  T. P. Ryan,et al.  A Preliminary Investigation of Maximum Likelihood Logistic Regression versus Exact Logistic Regression , 2002 .

[12]  D. G. Simpson,et al.  The Statistical Analysis of Discrete Data , 1989 .

[13]  P. McCullagh,et al.  Bias Correction in Generalized Linear Models , 1991 .

[14]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[15]  J M Taylor,et al.  The prognostic value of cellular and serologic markers in infection with human immunodeficiency virus type 1. , 1990, The New England journal of medicine.