A Preliminary Investigation of Maximum Likelihood Logistic Regression versus Exact Logistic Regression

Logistic regression is used by practitioners and researchers in many fields, but is undoubtedly used most frequently in medical and biostatistical applications. Maximum likelihood is generally the estimation method of choice, but we show that maximum likelihood can produce very poor results under certain conditions. Specifically, the poor performance of maximum likelihood in the case of rare events is known and we review research on this topic. We primarily examine the performance of maximum likelihood in the presence of near separation, which has apparently not been studied. Exact logistic regression is the logical alternative to maximum likelihood. We offer a comparison of the two methods of estimation.

[1]  John N. Pearson,et al.  Jit Manufacturing: a Survey of Implementations in Small and Large U.S. Manufacturers , 1999 .

[2]  D. Cox,et al.  Analysis of Binary Data (2nd ed.). , 1990 .

[3]  P. Beja,et al.  The use of sighting data to analyse Iberian lynx habitat and distribution , 1999 .

[4]  Eric R. Ziegel,et al.  Analysis of Binary Data (2nd ed.) , 1991 .

[5]  Yu Hen Hu,et al.  Logistic Regression in an Adaptive Web Cache , 1999, IEEE Internet Comput..

[6]  R. J. O'Hara Hines,et al.  Improved Added Variable and Partial Residual Plots for the Detection of Influential Observations in Generalized Linear Models , 1993 .

[7]  G. Barnard.  Must clinical trials be large? The interpretation of P-values and the combination of test results. , 1990, Statistics in medicine.

[8]  D. Collett,et al.  Modelling Binary Data , 1991 .

[9]  David R. Cox.  The analysis of binary data , 1970 .

[10]  J. S. Cramer,et al.  Predicitive performance of the binary logit model in unbalanced samples , 1999 .

[11]  A. Albert,et al.  On the existence of maximum likelihood estimates in logistic regression models , 1984 .

[12]  D. Tritchler.  An Algorithm for Exact Logistic Regression , 1984 .

[13]  Gary King,et al.  Explaining Rare Events in International Relations , 2001, International Organization.

[14]  Roderick J. A. Little,et al.  Testing the Equality of Two Independent Binomial Proportions , 1989 .

[15]  Cyrus R. Mehta,et al.  Efficient Monte Carlo Methods for Conditional Logistic Regression , 2000 .

[16]  Thomas J. Santner,et al.  A note on A. Albert and J. A. Anderson's conditions for the existence of maximum likelihood estimates in logistic regression models , 1986 .

[17]  Nitin R. Patel,et al.  Exact logistic regression: theory and examples. , 1995, Statistics in medicine.

[18]  Gary King,et al.  Logistic Regression in Rare Events Data , 2001, Political Analysis.