MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity

We present a new logic-based inference engine for natural language inference (NLI) called MonaLog, which is based on natural logic and the monotonicity calculus. In contrast to existing logic-based approaches, our system is intentionally designed to be as lightweight as possible, and operates using a small set of well-known (surface-level) monotonicity facts about quantifiers, lexical items and tokenlevel polarity information. Despite its simplicity, we find our approach to be competitive with other logic-based NLI models on the SICK benchmark. We also use MonaLog in combination with the current state-of-the-art model BERT in a variety of settings, including for compositional data augmentation. We show that MonaLog is capable of generating large amounts of high-quality training data for BERT, improving its accuracy on SICK.

[1]  R. Thomas McCoy,et al.  Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference , 2019, ACL.

[2]  Christopher D. Manning,et al.  Natural language inference , 2009 .

[3]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[4]  Katrin Erk,et al.  Representing Meaning with a Combination of Logical and Distributional Models , 2015, CL.

[5]  Valeria de Paiva,et al.  Textual Inference: getting logic from humans , 2017, IWCS.

[6]  Valeria C V de Paiva,et al.  WordNet for “ Easy ” Textual Inferences , 2018 .

[7]  Samuel R. Bowman,et al.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , 2017, NAACL.

[8]  Christopher D. Manning,et al.  A Phrase-Based Alignment Model for Natural Language Inference , 2008, EMNLP.

[9]  Rachel Rudinger,et al.  Hypothesis Only Baselines in Natural Language Inference , 2018, *SEMEVAL.

[10]  Marco Marelli,et al.  A SICK cure for the evaluation of compositional distributional semantic models , 2014, LREC.

[11]  Peter Clark,et al.  SciTaiL: A Textual Entailment Dataset from Science Question Answering , 2018, AAAI.

[12]  Christopher D. Manning,et al.  Modeling Semantic Containment and Exclusion in Natural Language Inference , 2008, COLING.

[13]  Pascual Martínez-Gómez,et al.  On-demand Injection of Lexical Knowledge for Recognising Textual Entailment , 2017, EACL.

[14]  Lawrence S. Moss,et al.  Probing Natural Language Inference Models through Semantic Fragments , 2020, AAAI.

[15]  Omer Levy,et al.  Annotation Artifacts in Natural Language Inference Data , 2018, NAACL.

[16]  Yorick Wilks,et al.  Natural language inference. , 1973 .

[17]  Christopher D. Manning,et al.  NaturalLI: Natural Logic Inference for Common Sense Reasoning , 2014, EMNLP.

[18]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[19]  Johan Bos,et al.  Recognising Textual Entailment with Logical Inference , 2005, HLT.

[20]  Pascual Martínez-Gómez,et al.  Acquisition of Phrase Correspondences Using Natural Deduction Proofs , 2018, NAACL.

[21]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[22]  Christopher D. Manning,et al.  An extended model of natural logic , 2009, IWCS.

[23]  Ido Dagan,et al.  A Confidence Model for Syntactically-Motivated Entailment Proofs , 2011, RANLP.

[24]  Stephen Pulman,et al.  Using the Framework , 1996 .

[25]  Carolyn Penstein Rosé,et al.  Stress Test Evaluation for Natural Language Inference , 2018, COLING.

[26]  Lasha Abzianidze,et al.  LangPro: Natural Language Theorem Prover , 2017, EMNLP.

[27]  Valeria de Paiva,et al.  Explaining Simple Natural Language Inference , 2019, LAW@ACL.

[28]  Pascual Martínez-Gómez,et al.  ccg2lambda: A Compositional Semantics System , 2016, ACL.

[29]  Hai Hu,et al.  Polarity Computations in Flexible Categorial Grammar , 2018, *SEM@NAACL-HLT.

[30]  Wenpeng Yin,et al.  Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching , 2017, EACL.

[31]  Valeria C V de Paiva,et al.  Correcting Contradictions , 2017 .

[32]  J. Benthem.  Essays in Logical Semantics , 1986 .

[33]  Hai Hu,et al.  Natural Language Inference with Monotonicity , 2019, IWCS.

[34]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[35]  Pascual Martínez-Gómez,et al.  Higher-order logical inference with compositional semantics , 2015, EMNLP.

[36]  Christopher Potts,et al.  A large annotated corpus for learning natural language inference , 2015, EMNLP.

[37]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[38]  Lasha Abzianidze,et al.  A Tableau Prover for Natural Logic and Language , 2015, EMNLP.

[39]  Malvina Nissim,et al.  The Meaning Factory: Formal Semantics for Recognizing Textual Entailment and Determining Semantic Similarity , 2014, *SEMEVAL.