A New Measure of Conditional Dependence for Causal Structural Learning

Measuring the dependencies among the variables of a network is of great interest to many disciplines. This paper studies the limitations of the existing dependencies measures such as their shortcomings in detecting direct influences or their lack of ability for group selection in order to have effective interventions and introduces a new statistical influence measure to overcome them. This measure is inspired by Dobrushin's coefficients and has been developed based on the paradigm that the conditional distribution of the variable of interest given all the direct causes will not change by intervening on other variables in the system. We show the advantageous of this measure over the related measures in the literature. Moreover, we establish the connection between our measure and the integral probability metric (IPM) that helps to develop estimators for our measure with lower complexity compared to the other relevant information theoretic based measures. At the end, we show the performance of this measure through a numerical simulation.

[1]  R. Dobrushin.  Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[2]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[3]  Jonas Peters,et al.  Causal inference by using invariant prediction: identification and confidence intervals , 2015, 1501.01332.

[4]  Steven A. Sloman,et al.  Causal reasoning through intervention , 2007 .

[5]  Richard Scheines,et al.  Constructing Bayesian Network Models of Gene Expression Networks from Microarray Data , 2000 .

[6]  Bernhard Schölkopf,et al.  Kernel-based Conditional Independence Test and Application in Causal Discovery , 2011, UAI.

[7]  Bernhard Schölkopf,et al.  Non-parametric estimation of integral probability metrics , 2010, 2010 IEEE International Symposium on Information Theory.

[8]  Daniel Polani,et al.  Information Flows in Causal Networks , 2008, Adv. Complex Syst..

[9]  C. Villani.  Topics in Optimal Transportation , 2003 .

[10]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[11]  Yuesheng Xu,et al.  Universal Kernels , 2006, J. Mach. Learn. Res..

[12]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[13]  Bernhard Schölkopf,et al.  A kernel-based causal learning algorithm , 2007, ICML '07.

[14]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[15]  Saburou Saitoh,et al.  Theory of Reproducing Kernels and Its Applications , 1988 .

[16]  Yuichiro Kitajima,et al.  Causation and Intervention in Algebraic Quantum Field Theory , 2010 .

[17]  P. Holland.  CAUSAL INFERENCE, PATH ANALYSIS AND RECURSIVE STRUCTURAL EQUATIONS MODELS , 1988 .

[18]  Le Song,et al.  A Kernel Statistical Test of Independence , 2007, NIPS.

[19]  Qing Wang,et al.  Divergence estimation of continuous distributions based on data-dependent partitions , 2005, IEEE Transactions on Information Theory.

[20]  J. Massey.  CAUSALITY, FEEDBACK AND DIRECTED INFORMATION , 1990 .

[21]  Moninder Singh,et al.  Construction of Bayesian network structures from data: A brief survey and an efficient algorithm , 1995, Int. J. Approx. Reason..

[22]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[23]  Moritz Grosse-Wentrup,et al.  Quantifying causal influences , 2012, 1203.6502.

[24]  C. Granger.  Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[25]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[26]  Nihat Ay,et al.  Geometric robustness theory and biological networks , 2006, Theory in Biosciences.

[27]  A. Müller.  Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[28]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[29]  Christopher Meek,et al.  Strong completeness and faithfulness in Bayesian networks , 1995, UAI.

[30]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[31]  Aapo Hyvärinen,et al.  Discovery of Non-gaussian Linear Causal Models using ICA , 2005, UAI.

[32]  C. Givens,et al.  A class of Wasserstein metrics for probability distributions. , 1984 .

[33]  Bernhard Schölkopf,et al.  Kernel Measures of Conditional Dependence , 2007, NIPS.

[34]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[35]  Yangbo He,et al.  Active Learning of Causal Networks with Intervention Experiments and Optimal Designs , 2008 .

[36]  Daphne Koller,et al.  Active Learning for Structure in Bayesian Networks , 2001, IJCAI.