A quasi-exact test for comparing two binomial proportions.

The use of the Fisher exact test for comparing two independent binomial proportions has spawned an extensive controversy in the statistical literature. Many critics have faulted this test for being highly conservative. Partly in response to such criticism, some statisticians have suggested the use of a modified, non-randomized version of this test, namely the mid-P-value test. This paper examines the actual type I error rates of this test. For both one-sided and two-sided tests, and for a wide range of sample sizes, we show that the actual levels of significance of the mid-P-test tend to be closer to the nominal level as compared with various classical tests. The computational effort required for the mid-P-test is no more than that needed for the Fisher exact test. Further, the basis for its modification is a natural adjustment for discreteness; thus the test easily generalizes to r x c contingency tables and other discrete data problems.

[1]  G. Barnard,et al.  On alleged gains in power from lower P-values. , 1989, Statistics in medicine.

[2]  Ralph B. D'Agostino,et al.  The Appropriateness of Some Common Procedures for Testing the Equality of Two Independent Binomial Populations , 1988 .

[3]  J. Overall,et al.  Small-sample tests for homogeneity of response probabilities in 2 X 2 contingency tables. , 1987, Psychological bulletin.

[4]  Jean D. Gibbons,et al.  P-values: Interpretation and Methodology , 1975 .

[5]  M. G. Haviland.  Yates's correction for continuity and the analysis of 2 x 2 contingency tables. , 1990, Statistics in medicine.

[6]  G. Upton.  A Comparison of Alternative Tests for the 2 Times 2 Comparative Trial , 1982 .

[7]  K. D. Tocher.  Extension of the Neyman-Pearson theory of tests to discontinuous variates. , 1950, Biometrika.

[8]  Samy Suissa,et al.  Exact Unconditional Sample Sizes for the 2 Times 2 Binomial Trial , 1985 .

[9]  E. S. Pearson,et al.  The choice of statistical tests illustrated on the interpretation of data classed in a 2 X 2 table. , 1947, Biometrika.

[10]  David R. Cox.  The analysis of binary data , 1970 .

[11]  Oscar Kempthorne,et al.  In dispraise of the exact test: reactions☆ , 1979 .

[12]  W. Dupont.  Sensitivity of Fisher's exact test to minor perturbations in 2 x 2 contingency tables. , 1986, Statistics in medicine.

[13]  M. Stone,et al.  The role of significance testing: some data with a message , 1969 .

[14]  D. Liddell.  Practical Tests of 2Times2 Contingency Tables , 1976 .

[15]  Choongrak Kim,et al.  Exact Properties of Some Exact Test Statistics for Comparing Two Binomial Proportions , 1990 .

[16]  H. O. Lancaster,et al.  Significance Tests in Discrete Distributions , 1961 .

[17]  G. Barnard.  Significance tests for 2 X 2 tables. , 1947, Biometrika.

[18]  M. Pagano,et al.  Algorithms for the Analysis of Several $2 \times 2$ Contingency Tables , 1983 .

[19]  D. Basu.  Discussion of Joseph Berkson's paper “in dispraise of the exact test” , 1979 .

[20]  George A. Milliken,et al.  A Nonrandomized Unconditional Test for Comparing Two Proportions in 2×2 Contingency Tables , 1977 .

[21]  Joseph Berkson,et al.  In dispraise of the exact test: Do the marginal totals of the 2X2 table contain relevant information respecting the table proportions? , 1978 .

[22]  D. A. Williams,et al.  Tests for differences between several small proportions , 1988 .

[23]  F. Yates,et al.  Tests of Significance for 2 × 2 Contingency Tables , 1984 .

[24]  K F Hirji,et al.  Exact inference for matched case-control studies. , 1988, Biometrics.

[25]  C. Lloyd.  Doubling the one-sided P-value in testing independence in 2 x 2 tables against a two-sided alternative. , 1988, Statistics in medicine.

[26]  Jean D. Gibbons,et al.  Concepts of Nonparametric Theory , 1981 .

[27]  Nitin R. Patel,et al.  A Network Algorithm for Performing Fisher's Exact Test in r × c Contingency Tables , 1983 .

[28]  W. R. Rice,et al.  A New Probability Model for Determining Exact P-Values for 2 x 2 Contingency Tables When Comparing Binomial Proportions , 1988 .

[29]  G. A. Barnard,et al.  THE MEANING OF A SIGNIFICANCE LEVEL , 1947 .

[30]  W. Conover.  Some Reasons for Not Using the Yates Continuity Correction on 2×2 Contingency Tables , 1974 .

[31]  Gregory Camilli,et al.  Testing for association in 2 × 2 contingency tables with very small sample sizes. , 1979 .

[32]  Nitin R. Patel,et al.  Computing Distributions for Exact Logistic Regression , 1987 .