Designing an adaptive attention mechanism for relation classification

Entity pair provide essential information for identifying relation type. Aiming at this characteristic, Position Feature is widely used in current relation classification systems to highlight the words close to them. However, semantic knowledge involved in entity pair has not been fully utilized. To overcome this issue, we propose an Entity-pair-based Attention Mechanism, which is specially designed for relation classification. Recently, attention mechanism significantly promotes the development of deep learning in NLP. Inspired by this, for specific instance(entity pair, sentence), the corresponding entity pair information is incorporated as prior knowledge to adaptively compute attention weights for generating sentence representation. Experimental results on SemEval-2010 Task 8 dataset show that our method outperforms most of the state-of-the-art models, without external linguistic features.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Jun Zhao,et al.  Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks , 2015, EMNLP.

[3]  Sanda M. Harabagiu,et al.  UTD: Classifying Semantic Relations by Combining Lexical and Semantic Resources , 2010, *SEMEVAL.

[4]  Ting Wang,et al.  Improving Twitter Retrieval by Exploiting Structural Information , 2012, AAAI.

[5]  Yueting Zhuang,et al.  Community-Based Question Answering via Heterogeneous Social Network Learning , 2016, AAAI.

[6]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[7]  Jun Zhao,et al.  A Joint Model for Question Answering over Multiple Knowledge Bases , 2016, AAAI.

[8]  Ming Yang,et al.  Bidirectional Long Short-Term Memory Networks for Relation Classification , 2015, PACLIC.

[9]  Bowen Zhou,et al.  Classifying Relations by Ranking with Convolutional Neural Networks , 2015, ACL.

[10]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[11]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[12]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[13]  Dong Wang,et al.  Relation Classification via Recurrent Neural Network , 2015, ArXiv.

[14]  François Goasdoué,et al.  Explaining Inconsistency-Tolerant Query Answering over Description Logic Knowledge Bases , 2016, AAAI.

[15]  Zhi Jin,et al.  Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths , 2015, EMNLP.

[16]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[17]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[18]  김동규,et al.  [서평]「Algorithms on Strings, Trees, and Sequences」 , 2000 .

[19]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[20]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[21]  Zhang Xiong,et al.  Multidimensional scaling based knowledge provision for new questions in community Question Answering systems , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[22]  Jacopo Urbani,et al.  Column-Oriented Datalog Materialization for Large Knowledge Graphs , 2016, AAAI.

[23]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[24]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[25]  Masafumi Hagiwara,et al.  A natural language processing neural network comprehending English , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[26]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[27]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[28]  Geoffrey E. Hinton,et al.  Grammar as a Foreign Language , 2014, NIPS.