Reasoning about Independence in Probabilistic Models of Relational Data

Bayesian networks leverage conditional independence to compactly encode joint probability distributions. Many learning algorithms exploit the constraints implied by observed conditional independencies to learn the structure of Bayesian networks. The rules of d-separation provide a theoretical and algorithmic framework for deriving conditional independence facts from model structure. However, this theory only applies to Bayesian networks. Many realworld systems, such as social or economic systems, are characterized by interacting heterogeneous entities and probabilistic dependencies that cross the boundaries of entities. Consequently, researchers have developed extensions to Bayesian networks that can represent these relational dependencies. We show that the theory of d-separation inaccurately infers conditional independence when applied directly to the structure of probabilistic models of relational data. We introduce relational d-separation, a theory for deriving conditional independence facts from relational models. We provide a new representation, the abstract ground graph, that enables a sound, complete, and computationally ecient method for answering d-separation queries about relational models, and we present empirical results that demonstrate eectiveness.

[1]  Boaz Lerner,et al.  Bayesian Network Structure Learning by Recursive Autonomy Identification , 2009, J. Mach. Learn. Res..

[2]  Ben Taskar,et al.  Rich probabilistic models for gene expression , 2001, ISMB.

[3]  Hassan Khosravi,et al.  Learning directed relational models with recursive dependencies , 2011, Machine Learning.

[4]  Brian J. Taylor,et al.  Learning Causal Models of Relational Domains , 2010, AAAI.

[5]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[6]  Mark W. Schmidt,et al.  Modeling Discrete Interventional Data using Directed Cyclic Graphical Models , 2009, UAI.

[7]  Foster J. Provost,et al.  Distribution-based aggregation for relational learning with identifier attributes , 2006, Machine Learning.

[8]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[9]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[10]  A. Gelman Scaling regression inputs by dividing by two standard deviations , 2008, Statistics in medicine.

[11]  Ben Taskar,et al.  Learning Probabilistic Models of Link Structure , 2003, J. Mach. Learn. Res..

[12]  Judea Pearl,et al.  Complete Identification Methods for the Causal Hierarchy , 2008, J. Mach. Learn. Res..

[13]  Peter A. Flach,et al.  Propositionalization approaches to relational data mining , 2001 .

[14]  Katerina Marazopoulou,et al.  A Sound and Complete Algorithm for Learning Causal Models from Relational Data , 2013, UAI.

[15]  Jane Jorgensen,et al.  Ecosystem Analysis Using Probabilistic Relational Modeling , 2003 .

[16]  Lise Getoor,et al.  Understanding tuberculosis epidemiology using structured statistical models , 2004, Artif. Intell. Medicine.

[17]  Thomas S. Richardson,et al.  Learning high-dimensional directed acyclic graphs with latent and selection variables , 2011, 1104.5617.

[18]  Judea Pearl,et al.  A Theory of Inferred Causation , 1991, KR.

[19]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[20]  Peter Spirtes,et al.  Directed Cyclic Graphical Representations of Feedback Models , 1995, UAI.

[21]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[22]  Dan Geiger,et al.  On the logic of causal models , 2013, UAI.

[23]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[24]  Jennifer Neville,et al.  Relational Dependency Networks , 2007, J. Mach. Learn. Res..

[25]  Luc De Raedt,et al.  Basic Principles of Learning Bayesian Logic Programs , 2008, Probabilistic Inductive Logic Programming.

[26]  Denver Dash,et al.  Restructuring Dynamic Causal Systems in Equilibrium , 2005, AISTATS.

[27]  Philip S. Yu,et al.  Relevance search in heterogeneous networks , 2012, EDBT '12.

[28]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[29]  Vaughn R. McKim,et al.  Causality in crisis? : statistical methods and the search for causal knowledge in the social sciences , 1998 .

[30]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[31]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[32]  Dan Geiger,et al.  Identifying independence in bayesian networks , 1990, Networks.

[33]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[34]  Peter A. Flach Knowledge Representation for Inductive Learning , 1999, ESCQARU.

[35]  Jennifer Neville,et al.  Leveraging relational autocorrelation with latent group models , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[36]  Ben Taskar,et al.  Probabilistic Relational Models , 2014, Encyclopedia of Social Network Analysis and Mining.

[37]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[38]  David Heckerman,et al.  Probabilistic Entity-Relationship Models, PRMs, and Plate Models , 2004 .

[39]  Kathryn B. Laskey MEBN: A language for first-order Bayesian knowledge bases , 2008, Artif. Intell..

[40]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[41]  John M. Winn,et al.  Causality with Gates , 2012, AISTATS.

[42]  Professor Dr. Bernhard Thalheim Entity-Relationship Modeling , 2000, Springer Berlin Heidelberg.

[43]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[44]  Jennifer Neville,et al.  Linkage and Autocorrelation Cause Feature Selection Bias in Relational Learning , 2002, ICML.

[45]  David Heckerman,et al.  Probabilistic Models for Relational Data , 2004 .

[46]  Wray L. Buntine Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[47]  Jiji Zhang,et al.  On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias , 2008, Artif. Intell..

[48]  Jin Tian,et al.  Finding Minimal D-separators , 1998 .

[49]  Tyler J VanderWeele,et al.  On causal inference in the presence of interference , 2012, Statistical methods in medical research.

[50]  Jin Tian,et al.  A general identification condition for causal effects , 2002, AAAI/IAAI.

[51]  Nir Friedman,et al.  Inferring Cellular Networks Using Probabilistic Graphical Models , 2004, Science.

[52]  Lise Getoor,et al.  Learning statistical models from relational data , 2011, SIGMOD '11.

[53]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[54]  Lise Getoor,et al.  From Instances to Classes in Probabilistic Relational Models , 2000, ICML 2000.

[55]  Frederick Eberhardt,et al.  Learning linear cyclic causal models with latent variables , 2012, J. Mach. Learn. Res..

[56]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[57]  Thomas S. Richardson,et al.  Causal Inference in the Presence of Latent Variables and Selection Bias , 1995, UAI.

[58]  Ben Taskar,et al.  Probabilistic Classification and Clustering in Relational Data , 2001, IJCAI.

[59]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[60]  Cosma Rohilla Shalizi,et al.  Homophily and Contagion Are Generically Confounded in Observational Social Network Studies , 2010, Sociological methods & research.

[61]  Sebastian Thrun,et al.  Bayesian Network Induction via Local Neighborhoods , 1999, NIPS.

[62]  Tom Heskes,et al.  A Logical Characterization of Constraint-Based Causal Discovery , 2011, UAI.

[63]  David Poole,et al.  Logical Generative Models for Probabilistic Reasoning about Existence, Roles and Identity , 2007, AAAI.

[64]  Thomas S. Richardson,et al.  A factorization criterion for acyclic directed mixed graphs , 2009, UAI.

[65]  P. Spirtes,et al.  Ancestral graph Markov models , 2002 .

[66]  Rina Dechter,et al.  Identifying Independencies in Causal Graphs with Feedback , 1996, UAI.

[67]  Jennifer Neville,et al.  Relational Learning with One Network: An Asymptotic Analysis , 2011, AISTATS.

[68]  Mathias Ekstedt,et al.  A probabilistic relational model for security risk analysis , 2010, Comput. Secur..

[69]  Marco Valtorta,et al.  Identifiability in Causal Bayesian Networks: A Sound and Complete Algorithm , 2006, AAAI.

[70]  Raghu Ramakrishnan,et al.  Database Management Systems , 1976 .

[71]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[72]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[73]  Christopher M. Bishop Latent Variable Models , 1998, Learning in Graphical Models.

[74]  Tom Heskes,et al.  A Bayesian Approach to Constraint Based Causal Inference , 2012, UAI.

[75]  M. Hudgens,et al.  Toward Causal Inference With Interference , 2008, Journal of the American Statistical Association.

[76]  Haytham Elghazel,et al.  An Experimental Comparison of Hybrid Algorithms for Bayesian Network Structure Learning , 2012, ECML/PKDD.

[77]  André Elisseeff,et al.  Using Markov Blankets for Causal Structure Learning , 2008, J. Mach. Learn. Res..

[78]  Richard Barker,et al.  CASE method - entity relationship modelling , 1990, Computer aided systems engineering.

[79]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[80]  Peter Spirtes,et al.  Introduction to Causal Inference , 2010, J. Mach. Learn. Res..

[81]  Judea Pearl,et al.  Causal networks: semantics and expressiveness , 2013, UAI.

[82]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.