Misspecification tests for binomial and beta‐binomial models

The IOS test of Presnell and Boss (J. Am. Stat. Assoc. 2004; 99(465):216-227) is a general-purpose goodness-of-fit test based on the ratio of in-sample and out-of-sample likelihoods. For large samples, the IOS statistic can be approximated by a multiplicative contrast between two estimates of the information matrix, and in this way the IOS test is connected to White's (Econometrica 1982; 50:1-26) information matrix test, or IM test, which is based directly on the difference of two estimates of the information matrix. In this paper, we compare the performance of IOS to that of the IM test and of other goodness-of-fit tests for binomial and beta-binomial models, in both examples and simulations. Our findings suggest that IOS is strongly competitive, not only against the IM test but also against tests designed for specific binomial and beta-binomial models.

[1]  A. Agresti.  Categorical data analysis , 1993 .

[2]  David R. Cox,et al.  Further Results on Tests of Separate Families of Hypotheses , 1962 .

[3]  Chris D. Orme,et al.  The small-sample performance of the information-matrix test , 1990 .

[4]  Xiaogang Su,et al.  Tree‐based model checking for logistic regression , 2007, Statistics in medicine.

[5]  W. Piegorsch,et al.  Assessing overdispersion and dose-response in the male dominant lethal assay. , 1992, Mutation research.

[6]  Haseman Jk,et al.  Analysis of dichotomous response data from certain toxicological experiments. , 1979 .

[7]  Biao Zhang.  A chi-squared goodness-of-fit test for logistic regression models based on case-control data , 1999 .

[8]  Biao Zhang,et al.  An information matrix test for logistic regression models based on case-control data , 2001 .

[9]  Richard L. Smith,et al.  Bootstrap goodness-of-fit test for the beta-binomial model , 2001 .

[10]  Dennis D. Boos,et al.  The IOS Test for Model Misspecification , 2004 .

[11]  J. Horowitz.  Bootstrap-based critical values for the information matrix test , 1994 .

[12]  Andrew Chesher,et al.  The information matrix test: Simplified calculation via a score test interpretation , 1983 .

[13]  R. Kass,et al.  Bayesian tests of extra‐Binomial variability , 2005, Statistics in medicine.

[14]  D. Hosmer,et al.  Goodness of fit tests for the multiple logistic regression model , 1980 .

[15]  Ronald Christensen,et al.  Significantly Insignificant F Tests , 2003 .

[16]  J. Qin,et al.  A goodness-of-fit test for logistic regression models based on case-control data , 1997 .

[17]  D. Cox.  Tests of Separate Families of Hypotheses , 1961 .

[18]  R. Tarone,et al.  Testing the goodness of fit of the binomial distribution , 1979 .

[19]  S P Brooks,et al.  Finite mixture models for proportions. , 1997, Biometrics.

[20]  Wei-Yin Loh,et al.  A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms , 2000, Machine Learning.

[21]  D. Hosmer,et al.  A comparison of goodness-of-fit tests for the logistic regression model. , 1997, Statistics in medicine.

[22]  Ross L. Prentice,et al.  Binary Regression Using an Extended Beta-Binomial Distribution, with Discussion of Correlation Induced by Covariate Measurement Errors , 1986 .

[23]  T. Stukel.  Generalized Logistic Models , 1988 .

[24]  J. B. Copas,et al.  Unweighted Sum of Squares Test for Proportions , 1989 .

[25]  S G Thompson,et al.  Comparing non-nested regression models. , 1995, Biometrics.

[26]  W W Piegorsch,et al.  On a Likelihood‐Based Goodness‐of‐Fit Test of the Beta‐Binomial Model , 2000, Biometrics.

[27]  Griffiths Da.  Maximum likelihood estimation for the beta-binomial distribution and an application to the household distribution of the total number of cases of a disease. , 1973 .

[28]  Williams Da,et al.  The analysis of binary responses from toxicological experiments involving reproduction and teratogenicity. , 1975 .

[29]  Tony Lancaster,et al.  THE COVARIANCE MATRIX OF THE INFORMATION MATRIX TEST , 1984 .

[30]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .