A comprehensive exploration of semantic relation extraction via pre-trained CNNs

Abstract Semantic relation extraction between entity pairs is a crucial task in information extraction from text. In this paper, we propose a new pre-trained network architecture for this task, and it is called the XM-CNN. The XM-CNN utilizes word embedding and position embedding information. It is designed to reinforce the contextual output from the MT-DNN K D pre-trained model. Our model effectively utilized an entity-aware attention mechanisms to detected the features and also adopts and applies more relation-specific pooling attention mechanisms applied to it. The experimental results show that the XM-CNN achieves state-of-the-art results on the SemEval-2010 task 8, and a thorough evaluation of the method is conducted.

[1]  Hamido Fujita,et al.  Computer Aided detection for fibrillations and flutters using deep convolutional neural network , 2019, Inf. Sci..

[2]  Gerhard Weikum,et al.  Combining linguistic and statistical analysis to extract relations from web documents , 2006, KDD '06.

[3]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[4]  Danushka Bollegala,et al.  Compositional approaches for representing relations between words: A comparative study , 2017, Knowl. Based Syst..

[5]  Pingyu Jiang,et al.  A deep learning approach for relationship extraction from interaction context in social manufacturing paradigm , 2016, Knowl. Based Syst..

[6]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[7]  Zhiyuan Liu,et al.  Relation Classification via Multi-Level Attention CNNs , 2016, ACL.

[8]  Yong Suk Choi,et al.  Semantic Relation Classification via Bidirectional LSTM Networks with Entity-aware Attention using Latent Entity Typing , 2019, Symmetry.

[9]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated diagnosis of congestive heart failure using ECG signals , 2018, Applied Intelligence.

[10]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[11]  Bowen Zhou,et al.  Classifying Relations by Ranking with Convolutional Neural Networks , 2015, ACL.

[12]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[13]  Tim Weninger,et al.  Discriminative predicate path mining for fact checking in knowledge graphs , 2015, Knowl. Based Syst..

[14]  Anjan Gudigar,et al.  Deep convolution neural network for accurate diagnosis of glaucoma using digital fundus images , 2018, Inf. Sci..

[15]  Kyung Sup Kwak,et al.  Transportation sentiment analysis using word embedding and ontology-based topic modeling , 2019, Knowl. Based Syst..

[16]  Dan Roth,et al.  Exploiting Background Knowledge for Relation Extraction , 2010, COLING.

[17]  Chiranjib Bhattacharyya,et al.  RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information , 2018, EMNLP.

[18]  Luke S. Zettlemoyer,et al.  Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations , 2011, ACL.

[19]  Yi Pan,et al.  Convolutional networks with cross-layer neurons for image recognition , 2018, Inf. Sci..

[20]  Sanda M. Harabagiu,et al.  UTD: Classifying Semantic Relations by Combining Lexical and Semantic Resources , 2010, *SEMEVAL.

[21]  Fang Kong,et al.  Exploiting Constituent Dependencies for Tree Kernel-Based Semantic Relation Extraction , 2008, COLING.

[22]  Jordi Turmo,et al.  Unsupervised Relation Extraction by Massive Clustering , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[23]  Dongyan Zhao,et al.  Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling , 2015, EMNLP.

[24]  Tingting He,et al.  Learning semantic representation with neural networks for community question answering retrieval , 2016, Knowl. Based Syst..

[25]  Jimmy J. Lin,et al.  Simple BERT Models for Relation Extraction and Semantic Role Labeling , 2019, ArXiv.

[26]  Sergey Brin,et al.  Extracting Patterns and Relations from the World Wide Web , 1998, WebDB.

[27]  Hamido Fujita,et al.  Word Sense Disambiguation: A comprehensive knowledge exploitation framework , 2020, Knowl. Based Syst..

[28]  Dong Wang,et al.  Relation Classification via Recurrent Neural Network , 2015, ArXiv.

[29]  Nanda Kambhatla,et al.  Combining Lexical, Syntactic, and Semantic Features with Maximum Entropy Models for Information Extraction , 2004, ACL.

[30]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[31]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[32]  Xiaodong Liu,et al.  Multi-Task Deep Neural Networks for Natural Language Understanding , 2019, ACL.

[33]  Xiaodong Liu,et al.  Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding , 2019, ArXiv.

[34]  Po-Sen Huang,et al.  Execution-Guided Neural Program Decoding , 2018, ArXiv.

[35]  Razvan C. Bunescu,et al.  A Shortest Path Dependency Kernel for Relation Extraction , 2005, HLT.

[36]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[37]  Andrew McCallum,et al.  Modeling Relations and Their Mentions without Labeled Text , 2010, ECML/PKDD.