Relational Logistic Regression: The Directed Analog of Markov Logic Networks

Logistic regression is a commonly used representation for aggregators in Bayesian belief networks when a child has multiple parents. In this paper we consider extending logistic regression to relational models, where we want to model varying populations and interactions among parents. In this paper, we first examine the representational problems caused by population variation. We show how these problems arise even in simple cases with a single parametrized parent, and propose a linear relational logistic regression which we show can represent arbitrary linear (in population size) decision thresholds, whereas the traditional logistic regression cannot. Then we examine representing interactions among the parents of a child node, and representing non-linear dependency on population size. We propose a multi-parent relational logistic regression which can represent interactions among parents and arbitrary polynomial decision thresholds. Finally, we show how other well-known aggregators can be represented using this relational logistic regression.

[1]  David Poole,et al.  The Independent Choice Logic and Beyond , 2008, Probabilistic Inductive Logic Programming.

[2]  Robert P. Goldman,et al.  From knowledge bases to decision models , 1992, The Knowledge Engineering Review.

[3]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[4]  Kristian Kersting,et al.  Exploiting Causal Independence in Markov Logic Networks: Combining Undirected and Directed Models , 2010, StarAI@AAAI.

[5]  D. Poole,et al.  Aggregation and Population Growth : The Relational Logistic Regression and Markov Logic Cases , 2012 .

[6]  Jennifer Neville,et al.  Using relational knowledge discovery to prevent securities fraud , 2005, KDD '05.

[7]  Matthew Richardson,et al.  Markov Logic , 2008, Probabilistic Inductive Logic Programming.

[8]  Tom. Mitchell.  GENERATIVE AND DISCRIMINATIVE CLASSIFIERS: NAIVE BAYES AND LOGISTIC REGRESSION Machine Learning , 2005 .

[9]  Shobha Venkataraman,et al.  Context-specific multiagent coordination and planning with factored MDPs , 2002, AAAI/IAAI.

[10]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[11]  Foster J. Provost,et al.  Distribution-based aggregation for relational learning with identifier attributes , 2006, Machine Learning.

[12]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[13]  George Danezis,et al.  SybilInfer: Detecting Sybil Nodes using Social Networks , 2009, NDSS.

[14]  Wray L. Buntine.  Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[15]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming , 2004, Probabilistic Inductive Logic Programming.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[18]  Kristian Kersting,et al.  Relational Logistic Regression: The Directed Analog of Markov Logic Networks , 2014, StarAI@AAAI.

[19]  Fahiem Bacchus,et al.  Towards Completely Lifted Search-based Probabilistic Inference , 2011, ArXiv.

[20]  Michael Beetz,et al.  Adaptive Markov Logic Networks: Learning Statistical Relational Models with Dynamic Parameters , 2010, ECAI.

[21]  Kristian Kersting,et al.  Population Size Extrapolation in Relational Probabilistic Modelling , 2014, SUM.

[22]  R. Smullyan.  First-Order Logic , 1968 .

[23]  Radford M. Neal.  Connectionist Learning of Belief Networks , 1992, Artif. Intell..

[24]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[25]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[26]  Michael Kaminsky,et al.  SybilGuard: defending against sybil attacks via social networks , 2006, SIGCOMM.

[27]  Mark W. Schmidt,et al.  On Sparse, Spectral and Other Parameterizations of Binary Probabilistic Models , 2012, AISTATS.

[28]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[29]  Michael Beetz,et al.  Extending Markov Logic to Model Probability Distributions in Relational Domains , 2007, KI.

[30]  Richard C. T. Lee,et al.  Symbolic logic and mechanical theorem proving , 1973, Computer science classics.

[31]  S. Cessie,et al.  Ridge Estimators in Logistic Regression , 1992 .

[32]  Jennifer Neville,et al.  Learning relational probability trees , 2003, KDD '03.

[33]  David Poole,et al.  Lifted Aggregation in Directed First-Order Probabilistic Models , 2009, IJCAI.

[34]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[35]  Jennifer Neville,et al.  Relational Dependency Networks , 2007, J. Mach. Learn. Res..

[36]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[37]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[38]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[39]  Nevin Lianwen Zhang,et al.  Exploiting Contextual Independence In Probabilistic Inference , 2011, J. Artif. Intell. Res..