Testing for Overdispersion in Poisson and Binomial Regression Models

Abstract In this article a method for obtaining tests for overdispersion with respect to a natural exponential family is derived. The tests are designed to be powerful against arbitrary alternative mixture models where only the first two moments of the mixed distribution are specified. Various tests for extra-Poisson and extra-binomial variation are obtained as special cases; the use of a particular test may be motivated by a consideration of the mechanism through which the overdispersion may arise. The common occurrence of extra-Poisson and extra-binomial variation has been noted by several authors. However, the Poisson and binomial models remain valid in many instances and, because of their simplicity and appeal, it is of real interest to ascertain when they apply. This paper develops a unifying theory for testing for overdispersion and generalizes tests previously derived, including those by Fisher (1950), Collings and Margolin (1985), and Prentice (1986). It also shows the Pearson statistic to be a sc...

[1]  Rory A. Fisher,et al.  The significance of deviations from expectation in a Poisson Series. , 1950 .

[2]  R F Potthoff,et al.  Testing for homogeneity. II. The Poisson distribution. , 1966, Biometrika.

[3]  P. A. P. Moran,et al.  An introduction to probability theory , 1968 .

[4]  G. J. Goodhardt,et al.  The Beta‐Binomial Model for Consumer Purchasing Behaviour , 1970 .

[5]  P. A. P. Moran,et al.  Maximum-likelihood estimation in non-standard conditions , 1971, Mathematical Proceedings of the Cambridge Philosophical Society.

[6]  Williams Da,et al.  The analysis of binary responses from toxicological experiments involving reproduction and teratogenicity. , 1975 .

[7]  Martin Crowder,et al.  Beta-binomial Anova for Proportions , 1978 .

[8]  R. Tarone,et al.  Testing the goodness of fit of the binomial distribution , 1979 .

[9]  D. A. Williams,et al.  Extra‐Binomial Variation in Logistic Linear Models , 1982 .

[10]  David Ruppert,et al.  Robust Estimation in Heteroscedastic Linear Models. , 1982 .

[11]  E. Frome.  The analysis of rates using Poisson regression models. , 1983, Biometrics.

[12]  David R. Cox,et al.  Some remarks on overdispersion , 1983 .

[13]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[14]  N. Breslow.  Extra‐Poisson Variation in Log‐Linear Models , 1984 .

[15]  Andrew Chesher,et al.  Testing for Neglected Heterogeneity , 1984 .

[16]  Factors Influencing Motor Insurance Rates , 1985 .

[17]  Barry H. Margolin,et al.  Testing Goodness of Fit for the Poisson Assumption When Observations are Not Identically Distributed , 1985 .

[18]  Martin Crowder,et al.  Gaussian Estimation for Correlated Binomial Data , 1985 .

[19]  D. Brillinger,et al.  The natural variability of vital rates and associated statistics. , 1986, Biometrics.

[20]  Dirk F. Moore,et al.  Asymptotic properties of moment estimators for overdispersed counts and proportions , 1986 .

[21]  Ross L. Prentice,et al.  Binary Regression Using an Extended Beta-Binomial Distribution, with Discussion of Correlation Induced by Covariate Measurement Errors , 1986 .

[22]  Lung-fei Lee,et al.  Specification Test for Poisson Regression Models , 1986 .

[23]  A. Cameron,et al.  Econometric models based on count data. Comparisons and applications of some estimators and tests , 1986 .

[24]  R. Carroll,et al.  Variance Function Estimation , 1987 .

[25]  D. A. Williams,et al.  Generalized Linear Model Diagnostics Using the Deviance and Single Case Deletions , 1987 .

[26]  B. Jørgensen.  Exponential Dispersion Models , 1987 .

[27]  J. Nelder,et al.  An extended quasi-likelihood function , 1987 .

[28]  J. Lawless.  Regression Methods for Poisson Process Data , 1987 .

[29]  Dirk F. Moore,et al.  Modelling the Extraneous Variance in the Presence of Extra‐Binomial Variation , 1987 .

[30]  J. Lawless.  Negative binomial and mixed Poisson regression , 1987 .

[31]  D. Ruppert,et al.  Transformation and Weighting in Regression , 1988 .

[32]  Marie Davidian,et al.  A Note on Extended Quasi-Likelihood , 1988 .

[33]  J. Lawless,et al.  Tests for Detecting Overdispersion in Poisson Regression Models , 1989 .

[34]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[35]  Gordon E. Willmot,et al.  A mixed poisson–inverse‐gaussian regression model , 1989 .

[36]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[37]  P. Trivedi,et al.  Overdispersion tests for truncated Poisson regression models , 1992 .

[38]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .