Open-Domain Contextual Link Prediction and its Complementarity with Entailment Graphs

An open-domain knowledge graph (KG) has entities as nodes and natural language relations as edges, and is constructed by extracting (subject, relation, object) triples from text. The task of open-domain link prediction is to infer missing relations in the KG. Previous work has used standard link prediction for the task. Since triples are extracted from text, we can ground them in the larger textual context in which they were originally found. However, standard link prediction methods only rely on the KG structure and ignore the textual context that each triple was extracted from. In this paper, we introduce the new task of opendomain contextual link prediction which has access to both the textual context and the KG structure to perform link prediction. We build a dataset for the task and propose a model for it. Our experiments show that context is crucial in predicting missing relations. We also demonstrate the utility of contextual link prediction in discovering context-independent entailments between relations, in the form of entailment graphs (EG), in which the nodes are the relations. The reverse holds too: contextindependent EGs assist in predicting relations in context.

[1]  Xavier Holt,et al.  Probabilistic Models of Relational Implication , 2019, ArXiv.

[2]  Rainer Gemulla,et al.  Can We Predict New Facts with Open Knowledge Graph Embeddings? A Benchmark for Open Link Prediction , 2020, ACL.

[3]  Graham Neubig,et al.  How Can We Know What Language Models Know? , 2019, Transactions of the Association for Computational Linguistics.

[4]  Sebastian Riedel,et al.  Language Models as Knowledge Bases? , 2019, EMNLP.

[5]  Jeffrey Ling,et al.  Matching the Blanks: Distributional Similarity for Relation Learning , 2019, ACL.

[6]  Omer Levy,et al.  Annotating Relation Inference in Context via Question Answering , 2016, ACL.

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Mohammad Javad Hosseini.  Unsupervised Learning of Relational Entailment Graphs from Text , 2021 .

[10]  Jonathan Berant,et al.  BERTese: Learning to Speak to BERT , 2021, EACL.

[11]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.

[12]  Martin Schmitt,et al.  Language Models for Lexical Inference in Context , 2021, EACL.

[13]  Daniel S. Weld,et al.  Fine-Grained Entity Recognition , 2012, AAAI.

[14]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[15]  Mark Steedman,et al.  Learning Typed Entailment Graphs with Global Soft Constraints , 2018, Transactions of the Association for Computational Linguistics.

[16]  Mark Steedman,et al.  Duality of Link Prediction and Entailment Graph Induction , 2019, ACL.

[17]  Omer Levy,et al.  Do Supervised Distributional Methods Really Learn Lexical Inference Relations? , 2015, NAACL.

[18]  Daniel S. Weld,et al.  Harvesting Parallel News Streams to Generate Paraphrases of Event Relations , 2013, EMNLP.

[19]  Ido Dagan,et al.  Global Learning of Focused Entailment Graphs , 2010, ACL.

[20]  Steven Schockaert,et al.  Inducing Relational Knowledge from BERT , 2019, AAAI.

[21]  Ido Dagan,et al.  Learning Entailment Rules for Unary Templates , 2008, COLING.

[22]  Mark Steedman,et al.  Multivalent Entailment Graphs for Question Answering , 2021, EMNLP.

[23]  Markus Krötzsch,et al.  Wikidata , 2014, Commun. ACM.

[24]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[25]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[26]  Gerhard Weikum,et al.  AIDA-light: High-Throughput Named-Entity Disambiguation , 2014, LDOW.

[27]  Noga Alon,et al.  Efficient Global Learning of Entailment Graphs , 2015, CL.

[28]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[29]  Thomas Wolf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[30]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[31]  Philippe Langlais,et al.  WiRe57 : A Fine-Grained Benchmark for Open Information Extraction , 2018, LAW@ACL.

[32]  Fabio Petroni,et al.  How Context Affects Language Models' Factual Predictions , 2020, AKBC.

[33]  Patrick Pantel,et al.  Discovery of inference rules for question-answering , 2001, Natural Language Engineering.

[34]  Ido Dagan,et al.  Global Learning of Typed Entailment Rules , 2011, ACL.

[35]  Mausam,et al.  CaRB: A Crowdsourced Benchmark for Open IE , 2019, EMNLP.

[36]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[37]  Timothy M. Hospedales,et al.  TuckER: Tensor Factorization for Knowledge Graph Completion , 2019, EMNLP.