Structure Learning for Markov Logic Networks

Most existing learning methods for Markov Logic Networks (MLNs) use batch training, which becomes computationally expensive and eventually infeasible for large datasets with thousands of training examples which may not even all fit in main memory. To address this issue, previous work has used online learning to train MLNs. However, they all assume that the model’s structure (set of logical clauses) is given, and only learn the model’s parameters. However, the input structure is usually incomplete, so it should also be updated. In this work, we present OSL—the first algorithm that performs both online structure and parameter learning for MLNs. Experimental results on two realworld datasets for natural-language field segmentation show that OSL outperforms systems that cannot revise structure.

[1]  Michael R. Genesereth,et al.  Logical foundations of artificial intelligence , 1987 .

[2]  Raymond J. Mooney,et al.  Learning Relations by Pathfinding , 1992, AAAI.

[3]  M. Cali,et al.  Inducing logic programs without explicit negative examples , 1995 .

[4]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Mark Craven,et al.  Combining Statistical and Relational Methods for Learning in Hypertext Domains , 1998, ILP.

[6]  C. Lee Giles,et al.  Autonomous citation matching , 1999, AGENTS '99.

[7]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[8]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[9]  Andrew McCallum,et al.  Efficiently Inducing Features of Conditional Random Fields , 2002, UAI.

[10]  James Theiler,et al.  Online Feature Selection using Grafting , 2003, ICML.

[11]  Thomas Hofmann,et al.  Support vector machine learning for interdependent and structured output spaces , 2004, ICML.

[12]  Dan Klein,et al.  Unsupervised Learning of Field Segmentation Models for Information Extraction , 2005, ACL.

[13]  Xavier Carreras,et al.  Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling , 2005, CoNLL.

[14]  Pedro M. Domingos,et al.  Learning the structure of Markov logic networks , 2005, ICML.

[15]  David Page,et al.  Mode Directed Path Finding , 2005, ECML.

[16]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[17]  Daphne Koller,et al.  Efficient Structure Learning of Markov Networks using L1-Regularization , 2006, NIPS.

[18]  Raymond J. Mooney,et al.  Mapping and Revising Markov Logic Networks for Transfer Learning , 2007, AAAI.

[19]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields for Relational Learning , 2007 .

[20]  Raymond J. Mooney,et al.  Bottom-up learning of Markov logic network structure , 2007, ICML '07.

[21]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[22]  Pedro M. Domingos,et al.  Joint Inference in Information Extraction , 2007, AAAI.

[23]  Stefano Ferilli,et al.  Discriminative Structure Learning of Markov Logic Networks , 2008, ILP.

[24]  Iván V. Meza,et al.  Collective Semantic Role Labelling with Markov Logic , 2008, CoNLL.

[25]  Gerson Zaverucha,et al.  Using the Bottom Clause and Mode Declarations on FOL Theory Revision from Examples , 2008, ILP.

[26]  Pedro M. Domingos,et al.  Markov Logic: An Interface Layer for Artificial Intelligence , 2009, Markov Logic: An Interface Layer for Artificial Intelligence.

[27]  Raymond J. Mooney,et al.  Learning to Disambiguate Search Queries from Short Sessions , 2009, ECML/PKDD.

[28]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[29]  Pedro M. Domingos,et al.  Learning Markov logic network structure via hypergraph lifting , 2009, ICML '09.

[30]  Raymond J. Mooney,et al.  Max-Margin Weight Learning for Markov Logic Networks , 2009, ECML/PKDD.

[31]  Pedro M. Domingos,et al.  Learning Markov Logic Networks Using Structural Motifs , 2010, ICML.

[32]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[33]  Eric P. Xing,et al.  Grafting-light: fast, incremental feature selection and structure learning of Markov random fields , 2010, KDD '10.

[34]  Fei-Fei Li,et al.  What, Where and Who? Telling the Story of an Image by Activity Classification, Scene Recognition and Object Categorization , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[35]  Raymond J. Mooney,et al.  Online Max-Margin Weight Learning for Markov Logic Networks , 2011, SDM.