Nonparametric Bayesian Networks

A convenient way of modelling complex interactions is by employing graphs or networks which correspond to conditional independence structures in an underlying statistical model. One main class of models in this regard are Bayesian networks, which have the drawback of making parametric assumptions. Bayesian nonparametric mixture models offer a possibility to overcome this limitation, but have hardly been used in combination with networks. This manuscript bridges this gap by introducing nonparametric Bayesian network models. We review (parametric) Bayesian networks, in particular Gaussian Bayesian networks, from a Bayesian perspective as well as nonparametric Bayesian mixture models. Afterwards these two modelling approaches are combined into nonparametric Bayesian networks. The new models are compared both to Gaussian Bayesian networks and to mixture models in a simulation study, where it turns out that the nonparametric network models perform favorably in non Gaussian situations. The new models are also applied to an example from systems biology, namely finding modules within the MAPK cascade.

[1]  Guido Consonni,et al.  Moment priors for Bayesian model choice with applications to directed acyclic graphs , 2011 .

[2]  F. Quintana,et al.  Bayesian clustering and product partition models , 2003 .

[3]  Lancelot F. James,et al.  Generalized weighted Chinese restaurant processes for species sampling mixture models , 2003 .

[4]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.

[5]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[6]  Sach Mukherjee,et al.  Network inference using informative priors , 2008, Proceedings of the National Academy of Sciences.

[7]  E. Zamir,et al.  Reverse engineering intracellular biochemical networks. , 2008, Nature chemical biology.

[8]  S. Ghosal,et al.  Kullback Leibler property of kernel mixture priors in Bayesian density estimation , 2007, 0710.2746.

[9]  Judea Pearl,et al.  An Algorithm for Deciding if a Set of Observed Independencies Has a Causal Explanation , 1992, UAI.

[10]  Marco Grzegorczyk,et al.  Improving the structure MCMC sampler for Bayesian networks by introducing a new edge reversal move , 2008, Machine Learning.

[11]  Andrea Ongaro,et al.  Discrete random probability measures: a general framework for nonparametric Bayesian inference☆ , 2004 .

[12]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[13]  Alexander J. Hartemink,et al.  Non-stationary dynamic Bayesian networks , 2008, NIPS.

[14]  B. Kholodenko,et al.  Negative feedback and ultrasensitivity can bring about oscillations in the mitogen-activated protein kinase cascades. , 2000, European journal of biochemistry.

[15]  P. Müller,et al.  Defining Predictive Probability Functions for Species Sampling Models. , 2013, Statistical science : a review journal of the Institute of Mathematical Statistics.

[16]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[17]  Marco Grzegorczyk,et al.  Non-stationary continuous dynamic Bayesian networks , 2009, NIPS.

[18]  V. Johnson,et al.  On the use of non‐local prior densities in Bayesian hypothesis tests , 2010 .

[19]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[20]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[21]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[22]  Robert L. Wolpert,et al.  Nonparametric Function Estimation Using Overcomplete Dictionaries , 2006 .

[23]  Abel Rodríguez,et al.  Sparse covariance estimation in heterogeneous samples. , 2010, Electronic journal of statistics.

[24]  Ali Shojaie,et al.  Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs. , 2009, Biometrika.

[25]  Eduardo Sontag,et al.  Untangling the wires: A strategy to trace functional interactions in signaling and gene networks , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[26]  T. Ferguson.  Some Developments of the Blackwell-MacQueen Urn Scheme , 2005 .

[27]  John M. Noble,et al.  Bayesian Networks: An Introduction , 2009 .

[28]  K. Ickstadt,et al.  Improved criteria for clustering based on the posterior similarity matrix , 2009 .

[29]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[30]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[31]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[32]  P. Green,et al.  Bayesian Model-Based Clustering Procedures , 2007 .

[33]  J. Pitman.  Combinatorial Stochastic Processes , 2006 .

[34]  Lancelot F. James,et al.  Posterior Analysis for Normalized Random Measures with Independent Increments , 2009 .

[35]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[36]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[37]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[38]  James G. Scott,et al.  Objective Bayesian model selection in Gaussian graphical models , 2009 .

[39]  J. Hopfield,et al.  From molecular to modular cell biology , 1999, Nature.

[40]  David Heckerman,et al.  Learning Gaussian Networks , 1994, UAI.

[41]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[42]  Marco Grzegorczyk,et al.  Modelling non-stationary gene regulatory processes with a non-homogeneous Bayesian network and the allocation sampler , 2008, Bioinform..

[43]  John Geweke,et al.  Interpretation and inference in mixture models: Simple MCMC works , 2007, Comput. Stat. Data Anal..

[44]  NobileAgostino,et al.  Bayesian finite mixtures with an unknown number of components , 2007 .

[45]  Katja Ickstadt,et al.  Bayesian Nonparametric Estimation of Continuous Monotone Functions with Applications to Dose–Response Analysis , 2009, Biometrics.

[46]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[47]  Michal Linial,et al.  Using Bayesian Networks to Analyze Expression Data , 2000, J. Comput. Biol..