Discriminative Markov Logic Network Structure Learning Based on Propositionalization and chi2-Test

In this paper we present a bottom-up discriminative algorithm to learn automatically Markov Logic Network structures. Our approach relies on a new propositionalization method that transforms a learning dataset into an approximative representation in the form of boolean tables, from which to construct a set of candidate clauses according to a χ2-test. To compute and choose clauses, we successively use two different optimization criteria, namely pseudo-log-likelihood (PLL) and conditional log-likelihood (CLL), in order to combine the efficiency of PLL optimization algorithms together with the accuracy of CLL ones. First experiments show that our approach outperforms the existing discriminative MLN structure learning algorithms.

[1]  Pedro M. Domingos,et al.  Learning Markov Logic Networks Using Structural Motifs , 2010, ICML.

[2]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[3]  Joost N. Kok,et al.  Knowledge Discovery in Databases: PKDD 2007, 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, September 17-21, 2007, Proceedings , 2007, PKDD.

[4]  Stefano Ferilli,et al.  Structure Learning of Markov Logic Networks through Iterated Local Search , 2008, ECAI.

[5]  Pedro M. Domingos,et al.  Efficient Weight Learning for Markov Logic Networks , 2007, PKDD.

[6]  Petra Perner,et al.  Advances in Data Mining , 2002, Lecture Notes in Computer Science.

[7]  Luc De Raedt,et al.  Logical and relational learning , 2008, Cognitive Technologies.

[8]  Michael J. Pazzani,et al.  Relational Clichés: Constraining Induction During Relational Learning , 1991, ML.

[9]  Luc De Raedt,et al.  Adaptive Bayesian Logic Programs , 2001, ILP.

[10]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[11]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[12]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[13]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[14]  Pedro M. Domingos,et al.  Sound and Efficient Inference with Probabilistic and Deterministic Dependencies , 2006, AAAI.

[15]  Matthew Richardson,et al.  Speeding Up Inference in Statistical Relational Learning by Clustering Similar Query Literals , 2009, ILP.

[16]  A. Agresti.  Categorical data analysis , 1993 .

[17]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[18]  Raymond J. Mooney,et al.  Learning Relations by Pathfinding , 1992, AAAI.

[19]  Pedro M. Domingos,et al.  Discriminative Training of Markov Logic Networks , 2005, AAAI.

[20]  Jennifer Neville,et al.  Dependency networks for relational data , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[21]  Raymond J. Mooney,et al.  Max-Margin Weight Learning for Markov Logic Networks , 2009, ECML/PKDD.

[22]  Peter A. Flach,et al.  Evaluation Measures for Multi-class Subgroup Discovery , 2009, ECML/PKDD.

[23]  Luc De Raedt,et al.  Logical and Relational Learning: From ILP to MRDM (Cognitive Technologies) , 2008 .

[24]  Luc De Raedt,et al.  Clausal Discovery , 1997, Machine Learning.

[25]  Raymond J. Mooney,et al.  Bottom-up learning of Markov logic network structure , 2007, ICML '07.

[26]  Stefano Ferilli,et al.  Discriminative Structure Learning of Markov Logic Networks , 2008, ILP.

[27]  Bart Selman,et al.  A general stochastic approach to solving problems with hard and soft constraints , 1996, Satisfiability Problem: Theory and Applications.

[28]  Vasant Honavar,et al.  Efficient Markov Network Structure Discovery using Independence Tests , 2006, SDM.

[29]  Pedro M. Domingos,et al.  Memory-Efficient Inference in Relational Domains , 2006, AAAI.

[30]  Pedro M. Domingos,et al.  Learning the structure of Markov logic networks , 2005, ICML.

[31]  Pedro M. Domingos,et al.  Bottom-Up Learning of Markov Network Structure , 2010, ICML.

[32]  Alípio Mário Jorge,et al.  Iterative Induction of Logic Programs, An approach to logic program synthesis from incomplete specifications , 1999, AI Commun..

[33]  Sriraam Natarajan,et al.  Speeding Up Inference in Markov Logic Networks by Preprocessing to Reduce the Size of the Resulting Grounded Network , 2009, IJCAI.

[34]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Pedro M. Domingos,et al.  Learning Markov logic network structure via hypergraph lifting , 2009, ICML '09.

[36]  Leon G. Higley,et al.  Forensic Entomology: An Introduction , 2009 .

[37]  Raymond J. Mooney,et al.  Discriminative structure and parameter learning for Markov logic networks , 2008, ICML '08.

[38]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[39]  Matthew Richardson,et al.  The Alchemy System for Statistical Relational AI: User Manual , 2007 .

[40]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[41]  Pedro M. Domingos,et al.  A General Method for Reducing the Complexity of Relational Inference and its Application to MCMC , 2008, AAAI.