Extracting chemical-protein interactions from biomedical literature via granular attention based recurrent neural networks

BACKGROUND AND OBJECTIVE The extraction of interactions between chemicals and proteins from biomedical literature is important for many biomedical tasks such as drug discovery and precision medicine. In the existing systems, the methods achieving competitive results are combined of several models or implemented in multi-stage, and they are challenged by high cost because numerous external features are employed. These problems can be avoided by deep learning algorithms, but the performance of the deep learning based models is limited by inadequate exploration of the information. Our goal is to devise a system to improve the performance of the automatic extraction between chemical entities and protein entities from biomedical literature. METHODS In this paper, we propose a model based on recurrent neural networks integrating granular attention mechanism. The granular attention can explore the inner information of the context vectors, which are represented in multiple dimensions that play different roles in the extraction of the interactions. Furthermore, we employ Swish activation function in the neural networks for the chemical-protein interactions extraction task for the first time. RESULTS The proposed method is evaluated on BioCreative VI chemical-protein track test corpus. The experimental results show that this method achieves an F-score of 65.14%, which is 1.04% higher than the state-of-the-art system. CONCLUSIONS The model synthesizing recurrent neural networks and granular attention mechanism, exploring the inner information of the context vectors, can improve the extraction performance without extra hand-crafted features. The experimental results demonstrate that the proposed model is promising for further study on the interaction extraction between chemicals and proteins.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[3]  Jaewoo Kang,et al.  Chemical–gene relation extraction using recursive neural network , 2018, Database J. Biol. Databases Curation.

[4]  A. Valencia,et al.  Information Retrieval and Text Mining Technologies for Chemistry. , 2017, Chemical reviews.

[5]  Ali Farhadi,et al.  Bidirectional Attention Flow for Machine Comprehension , 2016, ICLR.

[6]  Yoshua Bengio,et al.  Fine-grained attention mechanism for neural machine translation , 2018, Neurocomputing.

[7]  Yoshua Bengio,et al.  Context-dependent word representation for neural machine translation , 2016, Comput. Speech Lang..

[8]  John Boyle,et al.  Improving the learning of chemical-protein interactions from literature using transfer learning and specialized word embeddings , 2018, Database J. Biol. Databases Curation.

[9]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[10]  Jari Björne,et al.  Potent pairing: ensemble of long short-term memory networks and support vector machine for chemical-protein relation extraction , 2018, Database J. Biol. Databases Curation.

[11]  Andrew McCallum,et al.  Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction , 2018, NAACL.

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[14]  Tao Shen,et al.  DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding , 2017, AAAI.

[15]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[16]  Quoc V. Le,et al.  Searching for Activation Functions , 2018, arXiv.

[17]  Ting Liu,et al.  Aspect Level Sentiment Classification with Deep Memory Network , 2016, EMNLP.

[18]  Hongfang Liu,et al.  Extracting chemical–protein relations using attention-based neural networks , 2018, Database J. Biol. Databases Curation.

[19]  Alexandros Potamianos,et al.  Structural Attention Neural Networks for improved sentiment analysis , 2017, EACL.

[20]  Sepp Hochreiter,et al.  Self-Normalizing Neural Networks , 2017, NIPS.

[21]  Yifan Peng,et al.  Extracting chemical–protein relations with ensembles of SVM and deep learning models , 2018, Database J. Biol. Databases Curation.

[22]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[23]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[24]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.