Multi-view Inference for Relation Extraction with Uncertain Knowledge

Knowledge graphs (KGs) are widely used to facilitate relation extraction (RE) tasks. While most previous RE methods focus on leveraging deterministic KGs, uncertain KGs, which assign a confidence score for each relation instance, can provide prior probability distributions of relational facts as valuable external knowledge for RE models. This paper proposes to exploit uncertain knowledge to improve relation extraction. Specifically, we introduce ProBase, an uncertain KG that indicates to what extent a target entity belongs to a concept, into our RE architecture. We then design a novel multi-view inference framework to systematically integrate local context and global knowledge across three views: mention-, entityand concept-view. The experimental results show that our model achieves competitive performances on both sentenceand document-level relation extraction, which verifies the effectiveness of introducing uncertain knowledge and the multiview inference framework that we design.

[1]  Yongcheng Li,et al.  Neural Relation Classification with Text Descriptions , 2018, COLING.

[2]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[3]  Qiang Dong,et al.  HowNet - a hybrid language and knowledge resource , 2003, International Conference on Natural Language Processing and Knowledge Engineering, 2003. Proceedings. 2003.

[4]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[5]  Sophia Ananiadou,et al.  Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs , 2019, EMNLP.

[6]  Jason Weston,et al.  Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction , 2013, EMNLP.

[7]  Sophia Ananiadou,et al.  Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network , 2019, ACL.

[8]  Zhiyuan Liu,et al.  Neural Knowledge Acquisition via Mutual Attention Between Knowledge Graph and Text , 2018, AAAI.

[9]  Amina Kadry,et al.  Open Relation Extraction for Support Passage Retrieval: Merit and Open Issues , 2017, SIGIR.

[10]  Ning Ding,et al.  Chinese Relation Extraction with Multi-Grained Information and External Linguistic Knowledge , 2019, ACL.

[11]  Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) , 2019, NAACL-HLT.

[12]  Yizhou Sun,et al.  Embedding Uncertain Knowledge Graphs , 2018, AAAI.

[13]  Makoto Miwa,et al.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures , 2016, ACL.

[14]  Andrew McCallum,et al.  Row-less Universal Schema , 2016, AKBC@NAACL-HLT.

[15]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[16]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[17]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[18]  Hong Wang,et al.  Fine-tune Bert for DocRED with Two-step Process , 2019, ArXiv.

[19]  Wei Zhang,et al.  Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding , 2018, EMNLP.

[20]  Denilson Barbosa,et al.  Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction , 2019, NAACL.

[21]  Nigel Collier,et al.  A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction , 2019, NAACL-HLT.

[22]  Yifan He,et al.  Enriching Pre-trained Language Model with Entity Information for Relation Classification , 2019, CIKM.

[23]  Bo Li,et al.  Graph Enhanced Dual Attention Network for Document-Level Relation Extraction , 2020, COLING.

[24]  Wei Lu,et al.  Reasoning with Latent Structure Refinement for Document-Level Relation Extraction , 2020, ACL.

[25]  Linmei Hu,et al.  Improving Distantly-Supervised Relation Extraction with Joint Label Embedding , 2019, EMNLP.

[26]  Wei Fan,et al.  Cooperative Denoising for Distantly Supervised Relation Extraction , 2018, COLING.

[27]  Haixun Wang,et al.  Probase: a probabilistic taxonomy for text understanding , 2012, SIGMOD Conference.

[28]  Maosong Sun,et al.  DocRED: A Large-Scale Document-Level Relation Extraction Dataset , 2019, ACL.

[29]  Lei Wang,et al.  A Novel Document-Level Relation Extraction Method Based on BERT and Entity Information , 2020, IEEE Access.

[30]  Kezhi Mao,et al.  Improving Relation Extraction with Knowledge-attention , 2019, EMNLP.

[31]  Yizhou Sun,et al.  Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts , 2019, KDD.

[32]  Sophia Ananiadou,et al.  A Walk-based Model on Entity Graphs for Relation Extraction , 2018, ACL.

[33]  Josef Ruppenhofer,et al.  FrameNet II: Extended theory and practice , 2006 .

[34]  Rui Xie,et al.  Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data , 2019, ACL.

[35]  Bowen Zhou,et al.  Improved Neural Relation Detection for Knowledge Base Question Answering , 2017, ACL.