Hyper-$g$ priors for generalized linear models

We develop an extension of the classical Zellner's g-prior to generalized linear models. The prior on the hyperparameter g is handled in a flexible way, so that any continuous proper hyperprior f(g) can be used, giving rise to a large class of hyper-g priors. Connections with the literature are described in detail. A fast and accurate integrated Laplace approximation of the marginal likelihood makes inference in large model spaces feasible. For posterior parameter estimation we propose an efficient and tuning-free Metropolis-Hastings sampler. The methodology is illustrated with variable selection and automatic covariate transformation in the Pima Indians diabetes data set.

[1]  C. Holmes,et al.  Bayesian auxiliary variable models for binary and multinomial regression , 2006 .

[2]  Bin Yu,et al.  Model Selection and the Principle of Minimum Description Length , 2001 .

[3]  Robert Kohn,et al.  Variable Selection and Model Averaging in Semiparametric Overdispersed Generalized Linear Models , 2007, 0707.2158.

[4]  J. Berger,et al.  Optimal predictive model selection , 2004, math/0406464.

[5]  Zhihua Zhang,et al.  Posterior Consistency of the Silverman g-prior in Bayesian Model Choice , 2008, NIPS.

[6]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo Methods for Computing Bayes Factors , 2001 .

[7]  James G. Scott,et al.  Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem , 2010, 1011.2333.

[8]  G. Box,et al.  Transformation of the Independent Variables , 1962 .

[9]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[10]  L. Wasserman,et al.  A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion , 1995 .

[11]  Jean-Michel Marin,et al.  Bayesian Core: A Practical Approach to Computational Bayesian Statistics , 2010 .

[12]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[13]  D. Lindley,et al.  Approximate Bayesian methods , 1980 .

[14]  A. Dobra.  Variable selection and dependency networks for genomewide data. , 2009, Biostatistics.

[15]  Dani Gamerman,et al.  Sampling from the posterior distribution in generalized linear mixed models , 1997, Stat. Comput..

[16]  Yuzo Maruyama,et al.  A g-prior extension for p>n , 2008 .

[17]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[18]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[19]  David J. Nott,et al.  Approximating the marginal likelihood using copula , 2008, 0810.5474.

[20]  M. Steel,et al.  Benchmark Priors for Bayesian Model Averaging , 2001 .

[21]  P. McCullagh,et al.  Monograph on Statistics and Applied Probability , 1989 .

[22]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[23]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[24]  Joseph G Ibrahim,et al.  An Information Matrix Prior for Bayesian Analysis in Generalized Linear Models with High Dimensional Data. , 2009, Statistica Sinica.

[25]  K. Burnham,et al.  Model selection: An integral part of inference , 1997 .

[26]  J. Naylor,et al.  Applications of a Method for the Efficient Computation of Posterior Distributions , 1982 .

[27]  Stephen Senn.  Comment on "Harold Jeffreys's Theory of Probability Revisited" , 2010 .

[28]  Christian P. Robert,et al.  The Bayesian choice , 1994 .

[29]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[30]  Jonathan J. Forster,et al.  Default Bayesian model determination methods for generalised linear mixed models , 2010, Comput. Stat. Data Anal..

[31]  Edward I. George,et al.  Empirical Bayes vs. Fully Bayes Variable Selection , 2008 .

[32]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[33]  S. Raudenbush,et al.  Maximum Likelihood for Generalized Linear Models with Nested Random Effects via High-Order, Multivariate Laplace Approximation , 2000 .

[34]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[35]  Dean Phillips Foster,et al.  Calibration and Empirical Bayes Variable Selection , 1997 .

[36]  S. Chib,et al.  Marginal Likelihood From the Metropolis–Hastings Output , 2001 .

[37]  James O. Berger,et al.  Objective Bayesian Methods for Model Selection: Introduction and Comparison , 2001 .

[38]  M. West,et al.  Shotgun Stochastic Search for “Large p” Regression , 2007 .

[39]  Joseph G Ibrahim,et al.  Bayesian Variable Selection and Computation for Generalized Linear Models with Conjugate Priors. , 2008, Bayesian analysis.

[40]  Leonhard Held,et al.  Bayesian fractional polynomials , 2011, Stat. Comput..

[41]  Bin Yu,et al.  Minimum Description Length Model Selection Criteria for Generalized Linear Models , 2003 .

[42]  P. Royston,et al.  Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. , 1994 .

[43]  P. Dellaportas,et al.  Bayesian variable and link determination for generalised linear models , 2003 .

[44]  J. Ibrahim,et al.  Conjugate priors for generalized linear models , 2003 .

[45]  Saleh,et al.  Point Estimation and Confidence Set Estimation in a Parallelism Model: an Empirical Bayes Approach , 1991 .

[46]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[47]  R. W. Wedderburn,et al.  On the existence and uniqueness of the maximum likelihood estimates for certain generalized linear models , 1976 .

[48]  Edward I. George,et al.  ADAPTIVE BAYESIAN CRITERIA IN VARIABLE SELECTION FOR GENERALIZED LINEAR MODELS , 2007 .

[49]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[50]  Gene H. Golub,et al.  Calculation of Gauss quadrature rules , 1967, Milestones in Matrix Computation.