Montague Meets Markov: Deep Semantics with Probabilistic Logical Form

We combine logical and distributional representations of natural language meaning by transforming distributional similarity judgments into weighted inference rules using Markov Logic Networks (MLNs). We show that this framework supports both judging sentence similarity and recognizing textual entailment by appropriately adapting the MLN implementation of logical connectives. We also show that distributional phrase similarity, used as textual inference rules created on the fly, improves its performance.

[1]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[2]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[3]  Kristian Kersting,et al.  Exploiting Causal Independence in Markov Logic Networks: Combining Undirected and Directed Models , 2010, StarAI@AAAI.

[4]  Pedro M. Domingos,et al.  A Tractable First-Order Probabilistic Logic , 2012, AAAI.

[5]  Carlo Strapparava,et al.  Corpus-based and Knowledge-based Measures of Text Semantic Similarity , 2006, AAAI.

[6]  Andrew Hickl,et al.  Using Discourse Commitments to Recognize Textual Entailment , 2008, COLING.

[7]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[8]  Pedro M. Domingos,et al.  Markov Logic: An Interface Layer for Artificial Intelligence , 2009, Markov Logic: An Interface Layer for Artificial Intelligence.

[9]  Chris Callison-Burch,et al.  Reranking Bilingually Extracted Paraphrases Using Monolingual Distributional Similarity , 2011, GEMS.

[10]  Daoud Clarke,et al.  A Context-Theoretic Framework for Compositionality in Distributional Semantics , 2011, Computational Linguistics.

[11]  J. Friedman.  Stochastic gradient boosting , 2002 .

[12]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[13]  Iryna Gurevych,et al.  UKP: Computing Semantic Textual Similarity by Combining Multiple Content Similarity Measures , 2012, *SEMEVAL.

[14]  Mirella Lapata,et al.  Vector-based Models of Semantic Composition , 2008, ACL.

[15]  Matthew Richardson,et al.  The Alchemy System for Statistical Relational AI: User Manual , 2007 .

[16]  Johan Bos,et al.  Wide-Coverage Semantic Analysis with Boxer , 2008, STEP.

[17]  Katrin Erk,et al.  Integrating Logical Representations with Probabilistic Information using Markov Logic , 2011, IWCS.

[18]  W. Lowe,et al.  Towards a Theory of Semantic Space , 2001 .

[19]  Dekang Lin,et al.  DIRT – Discovery of Inference Rules from Text , 2001 .

[20]  Ido Dagan,et al.  Directional distributional similarity for lexical inference , 2010, Natural Language Engineering.

[21]  László Dezsö,et al.  Universal Grammar , 1981, Certainty in Action.

[22]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[23]  Jerry R. Hobbs,et al.  Interpretation as Abduction , 1993, Artif. Intell..

[24]  L. Ferro,et al.  MITRE ’ s Submissions to the EU Pascal RTE Challenge , 2005 .

[25]  Patrick Pantel,et al.  DIRT @SBT@discovery of inference rules from text , 2001, KDD '01.

[26]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[27]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[28]  Ido Dagan,et al.  Learning Entailment Rules for Unary Templates , 2008, COLING.

[29]  Ido Dagan,et al.  Knowledge and Tree-Edits in Learnable Entailment Proofs , 2011, TAC.

[30]  Dan Flickinger,et al.  Minimal Recursion Semantics: An Introduction , 2005 .

[31]  Katrin Erk,et al.  A Structured Vector Space Model for Word Meaning in Context , 2008, EMNLP.

[32]  R. Montague.  Formal philosophy; selected papers of Richard Montague , 1974 .

[33]  Eneko Agirre,et al.  SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity , 2012, *SEMEVAL.

[34]  Katrin Erk,et al.  A Formal Approach to Linking Logical Form and Vector-Space Lexical Semantics , 2014 .

[35]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[36]  Johan Bos,et al.  Recognising Textual Entailment with Logical Inference , 2005, HLT.

[37]  William B. Dolan,et al.  Collecting Highly Parallel Data for Paraphrase Evaluation , 2011, ACL.

[38]  James R. Curran,et al.  Parsing the WSJ Using CCG and Log-Linear Models , 2004, ACL.

[39]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[40]  Mehrnoosh Sadrzadeh,et al.  Experimental Support for a Categorical Compositional Distributional Model of Meaning , 2011, EMNLP.

[41]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[42]  James R. Curran,et al.  Dependency Hashing for n-best CCG Parsing , 2012, ACL.

[43]  Ido Dagan,et al.  Global Learning of Typed Entailment Rules , 2011, ACL.

[44]  Patrick Pantel,et al.  Discovery of inference rules for question-answering , 2001, Natural Language Engineering.

[45]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[46]  Stefan Thater,et al.  Contextualizing Semantic Representations Using Syntactically Enriched Vector Models , 2010, ACL.

[47]  Rajat Raina,et al.  Robust Textual Inference Via Learning and Abductive Reasoning , 2005, AAAI.