Context-Specific and Local Independence in Markovian Dependence Structures

Directed acyclic graphs (DAGs) have been established as one of the primary tools for characterizing dependencies and causality among variables in multivariate systems. However, it has also been recognized that DAGs may hide more nuanced forms of independence that are important for interpretation and operational efficiency of the dependence models. Such independencies are typically context-specific, meaning that a variable may lose its connection to another variable in a particular context determined by some other set of variables. Here we review context-specific independence in different classes of Markovian probability models both for static and spatially or temporally organized variables, including Bayesian networks, Markov networks, and higher-order Markov chains. The generality of the context-specific independence as a concept may spawn new ways to characterize dependence systems also beyond these traditional models, for example, in dependence logic.

[1]  Erich Grädel,et al.  Dependence and Independence , 2012, Stud Logica.

[2]  Jukka Corander,et al.  Labelled Graphical Models , 2003 .

[3]  Nevin Lianwen Zhang,et al.  On the Role of Context-Specific Independence in Probabilistic Inference , 1999, IJCAI.

[4]  Jukka Corander,et al.  Stratified Graphical Models - Context-Specific Independence in Graphical Models , 2013, 1309.6415.

[5]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[6]  Meir Feder,et al.  A universal finite memory source , 1995, IEEE Trans. Inf. Theory.

[7]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[8]  Ronald L. Rivest,et al.  Inferring Decision Trees Using the Minimum Description Length Principle , 1989, Inf. Comput..

[9]  M. Golumbic.  Algorithmic graph theory and perfect graphs , 1980 .

[10]  Vanessa Didelez,et al.  Graphical models for marked point processes based on local independence , 2007, 0710.5874.

[11]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[12]  JORMA RISSANEN,et al.  A universal data compression system , 1983, IEEE Trans. Inf. Theory.

[13]  John M. Noble,et al.  Bayesian Networks: An Introduction , 2009 .

[14]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[15]  David Poole,et al.  Probabilistic Partial Evaluation: Exploiting Rule Structure in Probabilistic Inference , 1997, IJCAI.

[16]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[17]  Jukka Corander,et al.  Sparse Markov Chains for Sequence Data , 2014 .

[18]  Nevin Lianwen Zhang,et al.  Exploiting Contextual Independence In Probabilistic Inference , 2011, J. Artif. Intell. Res..

[19]  M. Golumbic.  Algorithmic Graph Theory and Perfect Graphs (Annals of Discrete Mathematics, Vol 57) , 2004 .

[20]  Jukka Corander,et al.  Bayesian graphical model determination using decision theory , 2003 .

[21]  Jukka Corander,et al.  Labeled directed acyclic graphs: a generalization of context-specific independence in directed graphical models , 2013, Data Mining and Knowledge Discovery.

[22]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[23]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[24]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[25]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[26]  Sergio Bacallado,et al.  Bayesian analysis of variable-order, reversible Markov chains , 2011, 1105.2640.

[27]  Juha Kontinen,et al.  Axiomatizing first order consequences in dependence logic , 2012, Ann. Pure Appl. Log..

[28]  P. Bühlmann,et al.  Variable Length Markov Chains: Methodology, Computing, and Software , 2004 .

[29]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[30]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[31]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[32]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[33]  Jukka Corander,et al.  Context-specific independence in graphical log-linear models , 2014, Comput. Stat..