Bayesian Model Averaging and Model Search Strategies

In regression models, such as generalized linear models, there is often substantial prior uncertainty about the choice of covariates to include. Conceptually, the Bayesian paradigm can easily incorporate this form of model uncertainty by building an expanded model that includes all possible subsets of covariates. In Bayesian model averaging, predictive distributions or posterior distributions of quantities of interest are obtained as mixtures of the model-specific distributions weighted by the posterior model probabilities. A major difficulty in implementing this approach is that the number of models in the mixture is often so large that enumeration of all models is impossible and some type of search strategy is required to determine a subset of models to use. In the case of an orthonormal design, some computationally simple approximations to the posterior model probabilities are introduced. These are used to develop efficient methods for deterministic or stochastic sampling from high-dimensional model spaces.

[1]  Adrian F. M. Smith,et al.  A Bayesian CART algorithm , 1998 .

[2]  Adrian E. Raftery,et al.  Accounting for Model Uncertainty in Survival Analysis Improves Predictive Performance , 1995 .

[3]  Stephen P. Brooks,et al.  Convergence Assessment for Reversible Jump MCMC Simulations , 2007 .

[4]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[5]  L. Wasserman,et al.  Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio , 1995 .

[6]  P. Dellaportas,et al.  Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models , 1999 .

[7]  Jack Cuzick,et al.  Geographical and environmental epidemiology : methods for small-area studies , 1997 .

[8]  Michael I. Jordan.  Graphical Models , 1998 .

[9]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[10]  P. Dellaportas,et al.  Bayesian variable selection using the Gibbs sampler , 2000 .

[11]  J M Davis,et al.  Assessing the human health risk of atmospheric particles. , 1999, Novartis Foundation symposium.

[12]  Edward E. Leamer,et al.  Specification Searches: Ad Hoc Inference with Nonexperimental Data , 1980 .

[13]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[14]  Adrian E. Raftery,et al.  MODEL SELECTION AND ACCOUNTING FOR MODEL UNCERTAINTY IN LINEAR REGRESSION MODELS , 2007 .

[15]  Merlise A. Clyde,et al.  Accounting for Model Uncertainty in Poisson Regression Models: Particulate Matter and Mortality in B , 1997 .

[16]  Adrian E. Raftery,et al.  ABayesian Model Averaging in Proportional Hazard Models: Assessing Stroke Risk , 1996 .

[17]  Giovanni Parmigiani,et al.  Orthogonalizations and Prior Distributions for Orthogonalized Model Mixing , 1996 .

[18]  Walter R. Gilks,et al.  Bayesian model comparison via jump diffusions , 1995 .

[19]  H. Chipman,et al.  Bayesian CART Model Search , 1998 .

[20]  G Parmigiani,et al.  Protein construct storage: Bayesian variable selection and prediction with mixtures. , 1998, Journal of biopharmaceutical statistics.

[21]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .

[22]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[23]  R. Kohn,et al.  Nonparametric regression using Bayesian variable selection , 1996 .

[24]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[25]  M. Clyde,et al.  Multiple shrinkage and subset selection in wavelets , 1998 .

[26]  D. Madigan,et al.  A method for simultaneous variable selection and outlier identification in linear regression , 1996 .

[27]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[28]  Robert Kohn,et al.  Nonparametric Estimation of Irregular Functions with Independent or Autocorrelated Errors , 1998 .

[29]  A. Raftery.  Approximate Bayes factors and accounting for model uncertainty in generalised linear models , 1996 .

[30]  Edward I. George,et al.  Two Approaches to Bayesian Model Selection with Applications , 1996 .

[31]  E. George,et al.  APPROACHES FOR BAYESIAN VARIABLE SELECTION , 1997 .

[32]  Adrian E. Raftery,et al.  Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data , 2005, Bioinform..

[33]  B. Efron.  Double Exponential Families and Their Use in Generalized Linear Regression , 1986 .

[34]  Walter R. Gilks,et al.  Strategies for improving MCMC , 1995 .

[35]  Petros Dellaportas,et al.  On Bayesian model and variable selection using MCMC , 2002, Stat. Comput..

[36]  M. Clyde,et al.  Prediction via Orthogonalized Model Mixing , 1996 .

[37]  J. Geweke,et al.  Variable selection and model comparison in regression , 1994 .

[38]  J. Dickey.  The Weighted Likelihood Ratio, Linear Hypotheses on Normal Location Parameters , 1971 .

[39]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .