Exploring Causal Relationships with Streaming Features

Causal discovery is highly desirable in science and technology. In this paper, we study a new research problem of discovery of causal relationships in the context of streaming features, where the features steam in one by one. With a Bayesian network to represent causal relationships, we propose a novel algorithm called causal discovery from streaming features (CDFSF) which consists of a two-phase scheme. In the first phase, CDFSF dynamically discovers causal relationships between each feature seen so far with an arriving feature, while in the second phase CDFSF removes the false positives of each arrived feature from its current set of direct causes and effects. To improve the efficiency of CDFSF, using the symmetry properties between parents (causes) and children (effects) in a faithful Bayesian network, we present a variant of CDFSF, S-CDFSF. Experimental results validate our algorithms in comparison with the existing algorithms of causal relationship discovery.

[1]  Richard Scheines,et al.  Causation, Prediction, and Search, Second Edition , 2000, Adaptive computation and machine learning.

[2]  Charu C. Aggarwal,et al.  Addressing Concept-Evolution in Concept-Drifting Data Streams , 2010, 2010 IEEE International Conference on Data Mining.

[3]  Constantin F. Aliferis,et al.  HITON: A Novel Markov Blanket Algorithm for Optimal Variable Selection , 2003, AMIA.

[4]  Wai Lam,et al.  Probabilistic joint models incorporating logic and learning via structured variational approximation for information extraction , 2011, Knowledge and Information Systems.

[5]  Jiye Li,et al.  Finding best evidence for evidence-based best practice recommendations in health care: the initial decision support system design , 2011, Knowledge and Information Systems.

[6]  C. Aliferis,et al.  Algorithms for Large-Scale Local Causal Discovery and Feature Selection In the Presence Of Limited Sample Or Large Causal Neighbourhoods , 2002 .

[7]  Constantin F. Aliferis,et al.  Time and sample efficient discovery of Markov blankets and direct causal relations , 2003, KDD '03.

[8]  Qiang Shen,et al.  Learning Bayesian Network Equivalence Classes with Ant Colony Optimization , 2009, J. Artif. Intell. Res..

[9]  Constantin F. Aliferis,et al.  Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part II: Analysis and Extensions , 2010, J. Mach. Learn. Res..

[10]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[11]  Xindong Wu,et al.  Personalized News Filtering and Summarization on the Web , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[12]  Jing Zhou,et al.  Streamwise Feature Selection , 2006, J. Mach. Learn. Res..

[13]  Sebastian Thrun,et al.  Bayesian Network Induction via Local Neighborhoods , 1999, NIPS.

[14]  Constantin F. Aliferis,et al.  Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part I: Algorithms and Empirical Evaluation , 2010, J. Mach. Learn. Res..

[15]  Nir Friedman,et al.  Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm , 1999, UAI.

[16]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[17]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[18]  Naren Ramakrishnan,et al.  Discovering excitatory relationships using dynamic Bayesian networks , 2011, Knowledge and Information Systems.

[19]  Satoru Miyano,et al.  Optimal Search on Clustered Structural Constraint for Learning Bayesian Network Structure , 2010, J. Mach. Learn. Res..

[20]  David Maxwell Chickering,et al.  Large-Sample Learning of Bayesian Networks is NP-Hard , 2002, J. Mach. Learn. Res..

[21]  Hao Wang,et al.  Online Streaming Feature Selection , 2010, ICML.

[22]  Constantin F. Aliferis,et al.  An Algorithm For Generation of Large Bayesian Networks , 2003 .

[23]  James Theiler,et al.  Online Feature Selection using Grafting , 2003, ICML.

[24]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[25]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[26]  Michal Linial,et al.  Using Bayesian Networks to Analyze Expression Data , 2000, J. Comput. Biol..

[27]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[28]  Judea Pearl,et al.  A Theory of Inferred Causation , 1991, KR.

[29]  Qiang Ji,et al.  Efficient Structure Learning of Bayesian Networks using Constraints , 2011, J. Mach. Learn. Res..

[30]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[31]  Michael Ruogu Zhang,et al.  Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization. , 1998, Molecular biology of the cell.

[32]  Changhe Yuan,et al.  Learning Optimal Bayesian Networks Using A* Search , 2011, IJCAI.

[33]  Jesper Tegnér,et al.  Towards scalable and data efficient learning of Markov boundaries , 2007, Int. J. Approx. Reason..

[34]  J. Pearl.  Causal diagrams for empirical research , 1995 .

[35]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[36]  Linda C. van der Gaag,et al.  Bayesian Belief Networks: Odds and Ends , 1996, Comput. J..

[37]  Marek J. Druzdzel,et al.  Robust Independence Testing for Constraint-Based Learning of Causal Structure , 2002, UAI.

[38]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[39]  Constantin F. Aliferis,et al.  Causal Explorer: A Causal Probabilistic Network Learning Toolkit for Biomedical Discovery , 2003, METMBS.

[40]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.