Learning Relational Probabilistic Models from Partially Observed Data-Opening the Closed-World Assumption

Recent years have seen a surge of interest in learning the structure of Statistical Relational Learning (SRL) models that combine logic with probabilities. Most of these models apply the closed-world assumption i.e., whatever is not observed is false in the world. In this work, we consider the problem of learning the structure of SRL models in the presence of hidden data i.e. we open the closed-world assumption. We develop a functional-gradient boosting algorithm based on EM to learn the structure and parameters of the models simultaneously and apply it to learn different kinds of models – Relational Dependency Networks, Markov Logic Networks and relational policies. Our results in a variety of domains demonstrate that the algorithms can effectively learn with missing data.

[1]  Kristian Kersting,et al.  Learning Markov Logic Networks via Functional Gradient Boosting , 2011, 2011 IEEE 11th International Conference on Data Mining.

[2]  Jianfeng Hu,et al.  Learning Compact Markov Logic Networks with Decision Trees , 2011, ILP.

[3]  Kristian Kersting,et al.  Imitation Learning in Relational Domains: A Functional-Gradient Boosting Approach , 2011, IJCAI.

[4]  Kristian Kersting,et al.  Gradient-based boosting for statistical relational learning: The relational dependency network case , 2011, Machine Learning.

[5]  Pedro M. Domingos,et al.  Learning Markov Logic Networks Using Structural Motifs , 2010, ICML.

[6]  Pedro M. Domingos,et al.  Markov Logic: An Interface Layer for Artificial Intelligence , 2009, Markov Logic: An Interface Layer for Artificial Intelligence.

[7]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[8]  Pedro M. Domingos,et al.  Learning Markov logic network structure via hypergraph lifting , 2009, ICML '09.

[9]  Jennifer Neville,et al.  Pseudolikelihood EM for Within-network Relational Learning , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[10]  Zhi-Hua Zhou,et al.  Structure Learning of Probabilistic Relational Models from Incomplete Relational Data , 2007, ECML.

[11]  Raymond J. Mooney,et al.  Bottom-up learning of Markov logic network structure , 2007, ICML '07.

[12]  Manfred Jaeger,et al.  Parameter learning for relational Bayesian networks , 2007, ICML '07.

[13]  Jennifer Neville,et al.  Relational Dependency Networks , 2007, J. Mach. Learn. Res..

[14]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[15]  Pedro M. Domingos,et al.  Entity Resolution with Markov Logic , 2006, Sixth International Conference on Data Mining (ICDM'06).

[16]  Thomas G. Dietterich,et al.  Learning first-order probabilistic models with combining rules , 2005, Annals of Mathematics and Artificial Intelligence.

[17]  Tapani Raiko,et al.  "Say EM" for Selecting Probabilistic Models for Logical Sequences , 2005, UAI.

[18]  Thomas G. Dietterich,et al.  Training conditional random fields via gradient tree boosting , 2004, ICML.

[19]  Jennifer Neville,et al.  Learning relational probability trees , 2003, KDD '03.

[20]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[21]  David Maxwell Chickering,et al.  Dependency Networks for Inference, Collaborative Filtering, and Data Visualization , 2000, J. Mach. Learn. Res..

[22]  Taisuke Sato,et al.  Efficient EM Learning with Tabulation for Parameterized Logic Programs , 2000, Computational Logic.

[23]  Hendrik Blockeel,et al.  Top-Down Induction of First Order Logical Decision Trees , 1998, AI Commun..

[24]  Salvatore J. Stolfo,et al.  Toward Scalable Learning with Non-Uniform Class and Cost Distributions: A Case Study in Credit Card Fraud Detection , 1998, KDD.

[25]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[26]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .