On Shannon capacity and causal estimation

The problem of estimating causal relationships from purely observational data is studied in this paper. We observe samples from a pair of random variables (X,Y) and wish to estimate whether X causes Y or Y causes X. Any joint distribution can be factored as p<sub>X,Y</sub> = p<sub>X</sub> p<sub>Y|X</sub> = p<sub>Y</sub> p<sub>X|Y</sub> and therefore the “causal” direction cannot be inferred from the joint distribution without further assumptions. In this paper, we propose and study the utility of Shannon capacity as a metric for causal directionality estimation. This opens up several open questions and directions for future study.

[1]  Yanjun Han,et al.  Minimax Estimation of Functionals of Discrete Distributions , 2014, IEEE Transactions on Information Theory.

[2]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Gregory Valiant,et al.  Estimating the unseen: an n/log(n)-sample estimator for entropy and support size, shown optimal via new CLTs , 2011, STOC '11.

[4]  Bernhard Schölkopf,et al.  Nonlinear causal discovery with additive noise models , 2008, NIPS.

[5]  C. Granger.  Some recent development in a concept of causality , 1988 .

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  Sean C. Bendall,et al.  Conditional density-based analysis of T cell signaling in single-cell data , 2014, Science.

[8]  Qing Wang,et al.  Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances , 2009, IEEE Transactions on Information Theory.

[9]  A. Rényi.  On measures of dependence , 1959 .

[10]  Joaquin Quiñonero Candela,et al.  Counterfactual reasoning and learning systems: the example of computational advertising , 2013, J. Mach. Learn. Res..

[11]  Justin Dauwels.  NUMERICAL COMPUTATION OF THE CAPACITY OF CONTINUOUS MEMORYLESS CHANNELS , 2005 .

[12]  Bernhard Schölkopf,et al.  Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks , 2014, J. Mach. Learn. Res..

[13]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[14]  Moritz Grosse-Wentrup,et al.  Quantifying causal influences , 2012, 1203.6502.

[15]  Bernhard Schölkopf,et al.  Inferring deterministic causal relations , 2010, UAI.

[16]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[17]  Bernhard Schölkopf,et al.  Information-geometric approach to inferring causal directions , 2012, Artif. Intell..