Stratified Graphical Models - Context-Specific Independence in Graphical Models

Theory of graphical models has matured over more than three decades to provide the backbone for several classes of models that are used in a myriad of applications such as genetic mapping of diseases, credit risk evaluation, reliability and computer security, etc. Despite of their generic applicability and wide adoptance, the constraints imposed by undirected graphical models and Bayesian networks have also been recognized to be unnecessarily stringent under certain circumstances. This observation has led to the proposal of several generalizations that aim at more relaxed constraints by which the models can impose local or context-specific dependence structures. Here we consider an additional class of such models, termed as stratified graphical models. We develop a method for Bayesian learning of these models by deriving an analytical expression for the marginal likelihood of data under a specific subclass of decomposable stratified models. A non-reversible Markov chain Monte Carlo approach is further used to identify models that are highly supported by the posterior distribution over the model space. Our method is illustrated and compared with ordinary graphical models through application to several real and synthetic datasets.

[1]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[2]  John M. Noble,et al.  Bayesian Networks: An Introduction , 2009 .

[3]  Jukka Corander,et al.  Labelled Graphical Models , 2003 .

[4]  Jukka Corander,et al.  Parallell interacting MCMC for learning of topologies of graphical models , 2008, Data Mining and Knowledge Discovery.

[5]  Poul Svante.  Decomposable log-linear models , 2015 .

[6]  Søren Højsgaard,et al.  Statistical Inference in Context Specific Interaction Models for Contingency Tables , 2004 .

[7]  P. Green,et al.  Decomposable graphical Gaussian model determination , 1999 .

[8]  A. Dawid,et al.  Hyper Markov Laws in the Statistical Analysis of Decomposable Graphical Models , 1993 .

[9]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[10]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[11]  M. Golumbic.  Algorithmic graph theory and perfect graphs , 1980 .

[12]  Mikko Koivisto,et al.  Exact Bayesian Structure Discovery in Bayesian Networks , 2004, J. Mach. Learn. Res..

[13]  Sergio Bacallado,et al.  Bayesian analysis of variable-order, reversible Markov chains , 2011, 1105.2640.

[14]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[15]  P. Bühlmann,et al.  Variable Length Markov Chains: Methodology, Computing, and Software , 2004 .

[16]  D. Geiger,et al.  Stratified exponential families: Graphical models and model selection , 2001 .

[17]  Akimichi Takemura,et al.  Hierarchical subspace models for contingency tables , 2009, J. Multivar. Anal..

[18]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[19]  Jukka Corander,et al.  Bayesian graphical model determination using decision theory , 2003 .

[20]  Søren Højsgaard,et al.  Split models for contingency tables , 2003, Comput. Stat. Data Anal..

[21]  S. Haberman,et al.  The analysis of frequency data , 1974 .

[22]  Paolo Giudici,et al.  Improving Markov Chain Monte Carlo Model Search for Data Mining , 2004, Machine Learning.

[23]  Mats Gyllenberg,et al.  Bayesian model learning based on a parallel MCMC strategy , 2006, Stat. Comput..

[24]  JORMA RISSANEN,et al.  A universal data compression system , 1983, IEEE Trans. Inf. Theory.

[25]  Poul Svante Eriksen,et al.  Decomposable log-linear models , 2005 .

[26]  T. Speed,et al.  Markov Fields and Log-Linear Interaction Models for Contingency Tables , 1980 .

[27]  Meir Feder,et al.  A universal finite memory source , 1995, IEEE Trans. Inf. Theory.

[28]  D. Edwards,et al.  A fast procedure for model search in multidimensional contingency tables , 1985 .

[29]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[30]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[31]  P. Dellaportas,et al.  Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models , 1999 .