Archival Version including Appendicies : Experiments in Stochastic Computation for High-Dimensional Graphical Models

We discuss the implementation, development and performance of methods of stochastic computation in Gaussian graphical models. We view these methods from the perspective of high-dimensional model search, with a particular interest in the scalability with dimension of Markov chain Monte Carlo (MCMC) and other stochastic search methods. After reviewing the structure and context of undirected Gaussian graphical models and model uncertainty (covariance selection), we discuss prior specifications, including new priors over models, and then explore a number of examples using various methods of stochastic computation. Traditional MCMC methods are the point of departure for this experimentation; we then develop alternative stochastic search ideas and contrast this new approach with MCMC. Our examples range from low (12–20) to moderate (150) dimension, and combine simple synthetic examples with data analysis from gene expression studies. We conclude with comments about the need and potential for new computational methods in far higher dimensions, including constructive approaches to Gaussian graphical modeling and computation.

[1]  J. Dickey.  The Weighted Likelihood Ratio, Linear Hypotheses on Normal Location Parameters , 1971 .

[2]  J. M. Hammersley,et al.  Markov fields on finite graphs and lattices , 1971 .

[3]  N. Wermuth.  Model Search among Multiplicative Models , 1976 .

[4]  P. Diaconis,et al.  Conjugate Priors for Exponential Families , 1979 .

[5]  Charles R. Johnson,et al.  Positive definite completions of partial Hermitian matrices , 1984 .

[6]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[7]  A. Dawid,et al.  Hyper Markov Laws in the Statistical Analysis of Decomposable Graphical Models , 1993 .

[8]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[9]  Michael I. Jordan.  Graphical Models , 2003 .

[10]  P. Dellaportas,et al.  Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models , 1999 .

[11]  D. Madigan,et al.  Graphical Markov models in mul-tivariate analysis , 1999 .

[12]  P. Green,et al.  Decomposable graphical Gaussian model determination , 1999 .

[13]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[14]  Subir Ghosh,et al.  Multivariate analysis, design of experiments, and survey sampling , 2000 .

[15]  David Maxwell Chickering,et al.  Dependency Networks for Inference, Collaborative Filtering, and Data Visualization , 2000, J. Mach. Learn. Res..

[16]  S E Fienberg,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:Bounds for cell entries in contingency tables given marginal totals and decomposable graphs , 2000 .

[17]  Michal Linial,et al.  Using Bayesian Networks to Analyze Expression Data , 2000, J. Comput. Biol..

[18]  D. Heckerman,et al.  Dependency networks for inference , 2000 .

[19]  Michael I. Jordan,et al.  Efficient Stepwise Selection in Decomposable Models , 2001, UAI.

[20]  R. Spang,et al.  Predicting the clinical status of human breast cancer by using gene expression profiles , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Kevin Kin Foon Wong.  An efficient sampler for decomposable covariance selection models , 2002 .

[22]  W. Wong,et al.  Transitive functional annotation by shortest-path analysis of gene expression data , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[23]  A. Roverato.  Hyper Inverse Wishart Distribution for Non-decomposable Graphs and its Application to Bayesian Inference for Gaussian Graphical Models , 2002 .

[24]  G. Roberts,et al.  Bayesian Inference For Nondecomposable Graphical Gaussian Models. , 2003 .

[25]  José A. Gámez,et al.  Incremental compilation of Bayesian networks , 2002, UAI.

[26]  Steffen L. Lauritzen,et al.  Graphical Models for Genetic Analyses , 2003 .

[27]  R. Kohn,et al.  Efficient estimation of covariance selection models , 2003 .

[28]  Paolo Giudici,et al.  Improving Markov Chain Monte Carlo Model Search for Data Mining , 2004, Machine Learning.

[29]  A. Dobra.  Bayesian Covariance Selection ∗ , 2004 .

[30]  Paul P. Wang,et al.  Advances to Bayesian network inference for generating causal networks from observational biological data , 2004, Bioinform..

[31]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .