Feature-Inclusion Stochastic Search for Gaussian Graphical Models

We describe a serial algorithm called feature-inclusion stochastic search, or FINCS, that uses online estimates of edge-inclusion probabilities to guide Bayesian model determination in Gaussian graphical models. FINCS is compared to MCMC, to Metropolis-based search methods, and to the popular lasso; it is found to be superior along a variety of dimensions, leading to better sets of discovered models, greater speed and stability, and reasonable estimates of edge-inclusion probabilities. We illustrate FINCS on an example involving mutual-fund data, where we compare the model-averaged predictive performance of models discovered with FINCS to those discovered by competing methods.

[1]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[2]  B. D. Finetti,et al.  Bayesian inference and decision techniques : essays in honor of Bruno de Finetti , 1986 .

[3]  M. Frydenberg,et al.  Decomposition of maximum likelihood in mixed graphical interaction models , 1989 .

[4]  A. Dawid,et al.  Hyper Markov Laws in the Statistical Analysis of Decomposable Graphical Models , 1993 .

[5]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[6]  A. O'Hagan,et al.  Fractional Bayes factors for model comparison , 1995 .

[7]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[8]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  Dean Phillips Foster,et al.  Calibration and Empirical Bayes Variable Selection , 1997 .

[10]  P. Green,et al.  Decomposable graphical Gaussian model determination , 1999 .

[11]  Michael I. Jordan,et al.  Efficient Stepwise Selection in Decomposable Models , 2001, UAI.

[12]  R. Kohn,et al.  Efficient estimation of covariance selection models , 2003 .

[13]  Michael A. West,et al.  Archival Version including Appendicies : Experiments in Stochastic Computation for High-Dimensional Graphical Models , 2005 .

[14]  J. Berger,et al.  Optimal predictive model selection , 2004, math/0406464.

[15]  Barry W. Peyton,et al.  Maximum Cardinality Search for Computing Minimal Triangulations of Graphs , 2004, Algorithmica.

[16]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .

[17]  James O. Berger,et al.  Posterior model probabilities via path‐based pairwise priors , 2005 .

[18]  James G. Scott,et al.  An exploration of aspects of Bayesian multiple testing , 2006 .

[19]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[20]  Pinar Heggernes,et al.  A vertex incremental approach for maintaining chordality , 2006, Discret. Math..

[21]  M. West,et al.  Simulation of hyper-inverse Wishart distributions in graphical models , 2007 .

[22]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[23]  Michael A. West,et al.  Dynamic matrix-variate graphical models , 2007 .

[24]  M. West,et al.  Shotgun Stochastic Search for “Large p” Regression , 2007 .

[25]  Edward I. George,et al.  Empirical Bayes vs. Fully Bayes Variable Selection , 2008 .

[26]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[27]  D. Woodard,et al.  Conditions for Rapid and Torpid Mixing of Parallel and Simulated Tempering on Multimodal Distributions , 2009, 0906.2341.

[28]  James G. Scott,et al.  Objective Bayesian model selection in Gaussian graphical models , 2009 .

[29]  James G. Scott,et al.  Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem , 2010, 1011.2333.