Bayesian Inference in Sparse Gaussian Graphical Models

One of the fundamental tasks of science is to find explainable relationships between observed phenomena. One approach to this task that has received attention in recent years is based on probabilistic graphical modelli ng with sparsity constraints on model structures. In this paper, we describe two new approaches to Bayesian inference of sparse structures of Gaussian graphi cal models (GGMs). One is based on a simple modification of the cutting-edge bloc k Gibbs sampler for sparse GGMs, which results in significant computational gains in high dimensions. The other method is based on a specific construction of the Hamiltonian Monte Carlo sampler, which results in further significant im provements. We compare our fully Bayesian approaches with the popular regular isation-based graphical LASSO, and demonstrate significant advantages of the Ba yesi n treatment under the same computing costs. We apply the methods to a broa d range of simulated data sets, and a real-life financial data set.

[1]  Hao Wang,et al.  Efficient Gaussian graphical model determination under G-Wishart prior distributions , 2012 .

[2]  Larry A. Wasserman,et al.  The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs , 2009, J. Mach. Learn. Res..

[3]  Abel Rodríguez,et al.  Sparse covariance estimation in heterogeneous samples. , 2010, Electronic journal of statistics.

[4]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[5]  Nicholas Mitsakakis,et al.  A Metropolis-Hastings based method for sampling from the G-Wishart distribution in Gaussian graphical models , 2011 .

[6]  A. Roverato.  Hyper Inverse Wishart Distribution for Non-decomposable Graphs and its Application to Bayesian Inference for Gaussian Graphical Models , 2002 .

[7]  Carlos M. Carvalho,et al.  Simulation of Hyper-Inverse Wishart Distributions for Non-decomposable Graphs , 2010 .

[8]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[9]  Abel Rodriguez,et al.  Bayesian Inference for General Gaussian Graphical Models With Application to Multivariate Lattice Data , 2010, Journal of the American Statistical Association.

[10]  Adrian Dobra,et al.  Modeling Uncertainty in Macroeconomic Growth Determinants Using Gaussian Graphical Models , 2008 .

[11]  Mauro Piccioni,et al.  Independence Structure of Natural Conjugate Densities to Exponential Families and the Gibbs' Sampler , 2000 .

[12]  A. Atay-Kayis,et al.  A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models , 2005 .

[13]  Amos J. Storkey,et al.  Discriminative Mixtures of Sparse Latent Fields for Risk Management , 2012, AISTATS.

[14]  Shiqian Ma,et al.  Sparse Inverse Covariance Selection via Alternating Linearization Methods , 2010, NIPS.

[15]  Adrian Dobra,et al.  Computational Aspects Related to Inference in Gaussian Graphical Models With the G-Wishart Prior , 2011 .

[16]  Trevor Hastie,et al.  Applications of the lasso and grouped lasso to the estimation of sparse graphical models , 2010 .

[17]  A. Rollett,et al.  The Monte Carlo Method , 2004 .

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[20]  Stephen Gould,et al.  Projected Subgradient Methods for Learning Sparse Gaussians , 2008, UAI.

[21]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[22]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[23]  A. Dobra,et al.  Copula Gaussian graphical models and their application to modeling functional disability data , 2011, 1108.1680.

[24]  Katherine A. Heller,et al.  Bayesian and L1 Approaches to Sparse Unsupervised Learning , 2011, ICML 2012.

[25]  Faming Liang,et al.  A double Metropolis–Hastings sampler for spatial models with intractable normalizing constants , 2010 .