Bayesian Regression Structure Discovery

The general problem of statistical regression is concerned with the discovery of a relationship between y and a set of potential predictors x1, . . . , xp. Because y may be related only to an unknown subset of the potential predictors, especially when p is large, variable selection is also an inherent part of this problem. In this chapter, we describe two very different Bayesian approaches to this general problem. In one case, variable selection takes place in the context of a tightly specified parametric model. Here, priors may be used to guide the model search and posterior quantities of interest are evident. In the other case, a far more flexible model, essentially nonparametric, allows for the opportunity to discover richer structure in the data, but requires more subtle methods for inference. With simple examples, we show how this second approach allows for model-free variable selection, and further for model-free interaction detection, the discovery of when variables work together to influence the response.