Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction

Abstract Joint entity and relation extraction is an essential task in information extraction, which aims to extract all relational triples from unstructured text. However, few existing works consider possible relations information between entities before extracting them, which may lead to the fact that most of the extracted entities cannot constitute valid triples. In this paper, we propose a representation iterative fusion based on heterogeneous graph neural networks for relation extraction (RIFRE). We model relations and words as nodes on the graph and fuse the two types of semantic nodes by the message passing mechanism iteratively to obtain nodes representation that is more suitable for relation extraction tasks. The model performs relation extraction after nodes representation is updated. We evaluate RIFRE on two public relation extraction datasets: NYT and WebNLG. The results show that RIFRE can effectively extract triples and achieve state-of-the-art performance. 1 Moreover, RIFRE is also suitable for the relation classification task, and significantly outperforms the previous methods on SemEval 2010 Task 8 datasets.

[1]  Bowen Zhou,et al.  Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs , 2019, ACL.

[2]  Senlin Luo,et al.  Joint extraction of entities and relations by a novel end-to-end model with a double-pointer module , 2020, Neurocomputing.

[3]  Shashi Narayan,et al.  Creating Training Corpora for NLG Micro-Planners , 2017, ACL.

[4]  Yifan He,et al.  Enriching Pre-trained Language Model with Entity Information for Relation Classification , 2019, CIKM.

[5]  Claire Cardie,et al.  Going out on a limb: Joint Extraction of Entity Mentions and Relations without Dependency Trees , 2017, ACL.

[6]  Preslav Nakov,et al.  SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals , 2009, SEW@NAACL-HLT.

[7]  Jun Zhao,et al.  Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism , 2018, ACL.

[8]  Pengfei Liu,et al.  Heterogeneous Graph Neural Networks for Extractive Document Summarization , 2020, ACL.

[9]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[10]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[11]  Zhepei Wei,et al.  A Novel Cascade Binary Tagging Framework for Relational Triple Extraction , 2019, ACL.

[12]  Roy Schwartz,et al.  Knowledge Enhanced Contextual Word Representations , 2019, EMNLP/IJCNLP.

[13]  Kaiwen Zhang,et al.  Improving Graph Convolutional Networks Based on Relation-Aware Attention for End-to-End Relation Extraction , 2020, IEEE Access.

[14]  Xiaofei Zhou,et al.  A Relation-Specific Attention Network for Joint Entity and Relation Extraction , 2020, IJCAI.

[15]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[16]  Jeffrey Ling,et al.  Matching the Blanks: Distributional Similarity for Relation Learning , 2019, ACL.

[17]  Dan Roth,et al.  Exploiting Syntactico-Semantic Structures for Relation Extraction , 2011, ACL.

[18]  Yuxiang Xie,et al.  Heterogeneous graph neural networks for noisy few-shot relation classification , 2020, Knowl. Based Syst..

[19]  Thomas Wolf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[20]  Zhiyuan Liu,et al.  Relation Classification via Multi-Level Attention CNNs , 2016, ACL.

[21]  Jian Su,et al.  Exploring Various Knowledge in Relation Extraction , 2005, ACL.

[22]  Andrew McCallum,et al.  Modeling Relations and Their Mentions without Labeled Text , 2010, ECML/PKDD.

[23]  Bin Wang,et al.  Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy , 2020, ECAI.

[24]  Joan Bruna,et al.  Few-Shot Learning with Graph Neural Networks , 2017, ICLR.

[25]  Peng Zhou,et al.  Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme , 2017, ACL.

[26]  Tong Xu,et al.  Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction , 2020, IJCAI.

[27]  Makoto Miwa,et al.  Modeling Joint Entity and Relation Extraction with Table Representation , 2014, EMNLP.

[28]  Shizhu He,et al.  Learning the Extraction Order of Multiple Relational Facts in a Sentence with Reinforcement Learning , 2019, EMNLP.

[29]  Wei-Yun Ma,et al.  GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction , 2019, ACL.