A SYMPTOTIC C AUSAL I NFERENCE A P REPRINT

We investigate causal inference in the asymptotic regime as the number of variables n→∞ using an information-theoretic framework. We define structural entropy of a causal model in terms of its description complexity measured by the logarithmic growth rate, measured in bits, of all directed acyclic graphs (DAGs) on n variables, parameterized by the edge density d. Structural entropy yields non-intuitive predictions. If we randomly sample a DAG from the space of all models over n variables, as n→∞, in the range d ∈ (0, 1 8 ), almost surely D is a two-layer DAG! Semantic entropy quantifies the reduction in entropy where edges are removed by causal intervention. Semantic causal entropy is defined as the φ-divergence Dφ(P ‖ PS) between the observational distribution P and the interventional distribution PS , where a subset S of edges are intervened on to determine their causal influence. We compare the decomposability properties of semantic entropy for different choices of φ, including φ(t) = t log t (KL-divergence), φ = 12 ( √ t− 1) (squared Hellinger distance), and φ = 12 |t − 1| (total variation distance). We apply our framework to generalize a recently popular bipartite experimental design for studying causal inference on large datasets, where interventions are carried out on one set of variables (e.g., power plants, items in an online store), but outcomes are measured on a disjoint set of variables (residents near power plants, or shoppers). We generalize bipartite designs to k-partite designs, and describe an optimization framework for finding the optimal k-level DAG architecture for any value of d ∈ (0, 12 ). As d increases, a sequence of phase transitions occur over disjoint intervals of d, with deeper DAG architectures emerging as d→ 1 2 . We also give a quantitative bound on the number of samples needed to reliably test for average causal influence for a k-partite design.