Multivariate Bayesian variable selection and prediction

The multivariate regression model is considered with p regressors. A latent vector with p binary entries serves to identify one of two types of regression coefficients: those close to 0 and those not. Specializing our general distributional setting to the linear model with Gaussian errors and using natural conjugate prior distributions, we derive the marginal posterior distribution of the binary latent vector. Fast algorithms aid its direct computation, and in high dimensions these are supplemented by a Markov chain Monte Carlo approach to sampling from the known posterior distribution. Problems with hundreds of regressor variables become quite feasible. We give a simple method of assigning the hyperparameters of the prior distribution. The posterior predictive distribution is derived and the approach illustrated on compositional analysis of data involving three sugars with 160 near infrared absorbances as regressors.

[1]  D. Lindley.  The Choice of Variables in Multiple Regression , 1968 .

[2]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[3]  A. Dawid.  Some matrix-variate distribution theory: Notational considerations and a Bayesian application , 1981 .

[4]  Paul H. Garthwaite,et al.  Elicitation of Prior Distributions for Variable-Selection Problems in Regression , 1992 .

[5]  Philip J. Brown,et al.  Wavelength selection in multicomponent near‐infrared calibration , 1992 .

[6]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[7]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[8]  P. Diaconis,et al.  Gray codes for randomization procedures , 1994 .

[9]  R. Carroll.  Measurement, Regression, and Calibration , 1994 .

[10]  J. Geweke,et al.  Variable selection and model comparison in regression , 1994 .

[11]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[12]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[13]  Purushottam W. Laud,et al.  Predictive Model Selection , 1995 .

[14]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[15]  Hugh Chipman,et al.  Bayesian variable selection with related predictors , 1995, bayes-an/9510001.

[16]  James E. Bennett,et al.  The Bayesian modeling of covariates for population pharmacokinetic models , 1996 .

[17]  L. Gleser.  Measurement, Regression, and Calibration , 1996 .

[18]  R. Kohn,et al.  Nonparametric regression using Bayesian variable selection , 1996 .

[19]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[20]  E. George,et al.  APPROACHES FOR BAYESIAN VARIABLE SELECTION , 1997 .

[21]  T. Fearn,et al.  Bayesian wavelength selection in multicomponent analysis , 1998 .