A Recursive Markov Blanket-Based Approach to Causal Structure Learning

One of the main approaches for causal structure learning is constraint-based methods. These methods are particularly valued as they are guaranteed to asymptotically find a structure which is statistically equivalent to the ground truth. However, they may require exponentially large number of conditional independence (CI) tests in the number of variables of the system. In this paper, we propose a novel recursive constraint-based method for causal structure learning. The key idea of the proposed approach is to recursively use Markov blanket information in order to identify a variable that can be removed from the set of variables without changing the statistical relations among the remaining variables. Once such a variable is found, its neighbors are identified, the removable variable is removed, and the Markov blanket information of the remaining variables is updated. Our proposed approach reduces the required number of conditional independence tests for structure learning compared to the state of the art. We also provide a lower bound on the number of CI tests required by any constraint-based method. Comparing this lower bound to our achievable bound demonstrates the efficiency of our approach. We evaluate and compare the performance of the proposed method on both synthetic and real world structures against the state of the art.

[1]  Illtyd Trethowan Causality , 1938 .

[2]  Shuigeng Zhou,et al.  Recursively Learning Causal Structures Using Regression-Based Conditional Independence Test , 2019, AAAI.

[3]  Zhi Geng,et al.  A Recursive Method for Structural Learning of Directed Acyclic Graphs , 2008, J. Mach. Learn. Res..

[4]  Constantin F. Aliferis,et al.  Algorithms for Large Scale Markov Blanket Discovery , 2003, FLAIRS.

[5]  Sebastian Thrun,et al.  Bayesian Network Induction via Local Neighborhoods , 1999, NIPS.

[6]  Shunkai Fu,et al.  Markov Blanket based Feature Selection: A Review of Past Decade , 2010 .

[7]  Bernhard Schölkopf,et al.  Nonlinear causal discovery with additive noise models , 2008, NIPS.

[8]  André Elisseeff,et al.  Using Markov Blankets for Causal Structure Learning , 2008, J. Mach. Learn. Res..

[9]  Peter Bühlmann,et al.  Estimating High-Dimensional Directed Acyclic Graphs with the PC-Algorithm , 2007, J. Mach. Learn. Res..

[10]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[11]  Mtw,et al.  Computation, causation, and discovery , 2000 .

[12]  AmirEmad Ghassami,et al.  Interventional Experiment Design for Causal Structure Learning , 2019, ArXiv.

[13]  Dimitris Margaritis,et al.  Speculative Markov blanket discovery for optimal feature selection , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[14]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[15]  Bernhard Schölkopf,et al.  Learning causality and causality-related learning: some recent progress. , 2018, National science review.

[16]  Daphne Koller,et al.  Ordering-Based Search: A Simple and Effective Algorithm for Learning Bayesian Networks , 2005, UAI.

[17]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[18]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..

[19]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[20]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[21]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[22]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[23]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[24]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[25]  B. Bollobás The evolution of random graphs , 1984 .

[26]  Mathias Frisch,et al.  Causation and intervention , 2014 .

[27]  Constantin F. Aliferis,et al.  Towards Principled Feature Selection: Relevancy, Filters and Wrappers , 2003 .

[28]  Christopher Meek,et al.  Causal inference and causal explanation with background knowledge , 1995, UAI.