Penalized loss functions for Bayesian model comparison.

The deviance information criterion (DIC) is widely used for Bayesian model comparison, despite the lack of a clear theoretical foundation. DIC is shown to be an approximation to a penalized loss function based on the deviance, with a penalty derived from a cross-validation argument. This approximation is valid only when the effective number of parameters in the model is much smaller than the number of independent observations. In disease mapping, a typical application of DIC, this assumption does not hold and DIC under-penalizes more complex models. Another deviance-based loss function, derived from the same decision-theoretic framework, is applied to mixture models, which have previously been considered an unsuitable application for DIC.

[1]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[2]  Tony O’Hagan.  Bayes factors , 2006 .

[3]  C. Robert,et al.  Deviance information criteria for missing data models , 2006 .

[4]  A. Linde.  Comment on article by Celeux et al. , 2006 .

[5]  Michael J Daniels,et al.  Longitudinal profiling of health care units based on continuous and discrete patient outcomes. , 2005, Biostatistics.

[6]  Bruce K Armstrong,et al.  Lung cancer rate predictions using generalized additive models. , 2005, Biostatistics.

[7]  James S Hodges,et al.  Generalized spatial structural equation models. , 2005, Biostatistics.

[8]  A. Linde.  DIC in variable selection , 2005 .

[9]  I. Katz,et al.  Using a Bayesian latent growth curve model to identify trajectories of positive affect and negative events following myocardial infarction. , 2005, Biostatistics.

[10]  Angelika van der Linde,et al.  On the association between a random parameter and an observable , 2004 .

[11]  Sw. Banerjee,et al.  Hierarchical Modeling and Analysis for Spatial Data , 2003 .

[12]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[13]  D J Spiegelhalter,et al.  Approximate cross‐validatory predictive checks in disease mapping models , 2003, Statistics in medicine.

[14]  David J. Spiegelhalter,et al.  WinBUGS user manual version 1.4 , 2003 .

[15]  Jouko Lampinen,et al.  Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities , 2002, Neural Computation.

[16]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[17]  Aki Vehtari,et al.  Discussion on the paper by Spiegelhalter, Best, Carlin and van der Linde , 2002 .

[18]  Aki Vehtari,et al.  Bayesian model assessment and selection using expected utilities , 2001 .

[19]  José M Bernardo and Adrian F M Smith,et al.  BAYESIAN THEORY , 2008 .

[20]  C. Robert,et al.  Computational and Inferential Difficulties with Mixture Posterior Distributions , 2000 .

[21]  M. Doll,et al.  Molecular genetics and epidemiology of the NAT1 and NAT2 acetylation polymorphisms. , 2000, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[22]  Alan E. Gelfand,et al.  Model choice: A minimum posterior predictive loss approach , 1998, AISTATS.

[23]  N. G. Best,et al.  Bayesian deviance the e ective number of parameters and the comparison of arbitrarily complex models , 1998 .

[24]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[25]  Mario Peruggia,et al.  On the variability of case-deletion importance sampling weights in the Bayesian linear model , 1997 .

[26]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[27]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[28]  Radford M. Neal.  Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[29]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[30]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[31]  A. O'Hagan,et al.  Fractional Bayes factors for model comparison , 1995 .

[32]  Purushottam W. Laud,et al.  Predictive Model Selection , 1995 .

[33]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[34]  Y. Bechtel,et al.  A population and family study N‐acetyltransferase using caffeine urinary metabolites , 1993, Clinical pharmacology and therapeutics.

[35]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[36]  M. Aitkin.  Posterior Bayes Factors , 1991 .

[37]  J. Besag,et al.  Bayesian image restoration, with two applications in spatial statistics , 1991 .

[38]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[39]  D. Clayton,et al.  Empirical Bayes estimates of age-standardized relative risks for use in disease mapping. , 1987, Biometrics.

[40]  B. Efron.  How Biased is the Apparent Error Rate of a Prediction Rule , 1986 .

[41]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[42]  J. Bernardo.  Expected Information as Expected Utility , 1979 .

[43]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[44]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[45]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .