Convolutional LSTM Network with Hierarchical Attention for Relation Classification in Clinical Texts

Identifying relation from clinical texts is a complex and challenging task due to the specific biomedical knowledge. Existing methods for this work generally have the misclassification problem caused by sample class imbalance. In this paper, we propose a hierarchical attention-based convolutional long short-term memory (ConvLSTM) network model to solve this problem. We construct a sentence as multi-dimensional hierarchical sequence and directly learn local and global context information by a single-layer ConvLSTM network. Besides, a hierarchical attention-based pooling is built to capture the parts of a sentence that are relevant with the target semantic relation. Experiments on the 2010 i2b2/VA relation dataset show that our model outperforms several previous state-of-the-art models without relying on any external features.

[1]  Yan Yang,et al.  DeepSTCL: A Deep Spatio-temporal ConvLSTM for Travel Demand Prediction , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[2]  Hongfang Liu,et al.  Journal of Biomedical Informatics , 2022 .

[3]  Sunil Kumar Sahu,et al.  Learning local and global contexts using a convolutional recurrent network model for relation classification in biomedical text , 2017, CoNLL.

[4]  Jianzhong Qiao,et al.  Relation Classification via Target-Concentrated Attention CNNs , 2017, ICONIP.

[5]  Makoto Miwa,et al.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures , 2016, ACL.

[6]  Jian Huang,et al.  End-to-End Continuous Emotion Recognition from Video Using 3D Convlstm Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Rui Dai,et al.  Classifying medical relations in clinical text via convolutional neural networks , 2018, Artif. Intell. Medicine.

[8]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[9]  Heng Ji,et al.  A Dependency-Based Neural Network for Relation Classification , 2015, ACL.

[10]  Sanda M. Harabagiu,et al.  Automatic extraction of relations between medical concepts in clinical texts , 2011, J. Am. Medical Informatics Assoc..

[11]  Bowen Zhou,et al.  Classifying Relations by Ranking with Convolutional Neural Networks , 2015, ACL.

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[14]  Sunil Kumar Sahu,et al.  Relation extraction from clinical texts using domain invariant convolutional neural network , 2016, BioNLP@ACL.

[15]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[16]  Zhi Jin,et al.  Improved relation classification by deep recurrent neural networks with data augmentation , 2016, COLING.

[17]  Peter Szolovits,et al.  Bridging semantics and syntax with graph algorithms - state-of-the-art of extracting biomedical relations , 2017, Briefings Bioinform..

[18]  Sanda M. Harabagiu,et al.  UTD: Classifying Semantic Relations by Combining Lexical and Semantic Resources , 2010, *SEMEVAL.

[19]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[20]  Sunil Kumar Sahu,et al.  Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network , 2017, J. Biomed. Informatics.

[21]  Dongyan Zhao,et al.  Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling , 2015, EMNLP.

[22]  Ming Yang,et al.  Bidirectional Long Short-Term Memory Networks for Relation Classification , 2015, PACLIC.

[23]  Yu Cheng,et al.  Segment convolutional neural networks (Seg-CNNs) for classifying relations in clinical notes , 2018, J. Am. Medical Informatics Assoc..

[24]  Razvan C. Bunescu,et al.  A Shortest Path Dependency Kernel for Relation Extraction , 2005, HLT.

[25]  Rui Dai,et al.  Convolutional Gated Recurrent Units for Medical Relation Classification , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[26]  Joel D. Martin,et al.  Detecting concept relations in clinical text: Insights from a state-of-the-art model , 2013, J. Biomed. Informatics.

[27]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[28]  Xiao Sun,et al.  Multichannel Convolutional Neural Network for Biological Relation Extraction , 2016, BioMed research international.

[29]  Yanyun Zhao,et al.  A spatio-temporal deep architecture for surveillance event detection based on ConvLSTM , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[30]  Houfeng Wang,et al.  Bidirectional Recurrent Convolutional Neural Network for Relation Classification , 2016, ACL.

[31]  Xiaolong Wang,et al.  Drug-Drug Interaction Extraction via Convolutional Neural Networks , 2016, Comput. Math. Methods Medicine.

[32]  Zhiyuan Liu,et al.  Relation Classification via Multi-Level Attention CNNs , 2016, ACL.

[33]  Ngoc Thang Vu,et al.  Combining Recurrent and Convolutional Neural Networks for Relation Classification , 2016, NAACL.