Biomedical document triage using a hierarchical attention-based capsule network

Background Biomedical document triage is the foundation of biomedical information extraction, which is important to precision medicine. Recently, some neural networks-based methods have been proposed to classify biomedical documents automatically. In the biomedical domain, documents are often very long and often contain very complicated sentences. However, the current methods still find it difficult to capture important features across sentences. Results In this paper, we propose a hierarchical attention-based capsule model for biomedical document triage. The proposed model effectively employs hierarchical attention mechanism and capsule networks to capture valuable features across sentences and construct a final latent feature representation for a document. We evaluated our model on three public corpora. Conclusions Experimental results showed that both hierarchical attention mechanism and capsule networks are helpful in biomedical document triage task. Our method proved itself highly competitive or superior compared with other state-of-the-art methods.

[1]  Zhiyong Lu,et al.  Text Mining Genotype-Phenotype Relationships from Biomedical Literature for Database Curation and Precision Medicine , 2016, PLoS Comput. Biol..

[2]  Min Yang,et al.  Investigating Capsule Networks with Dynamic Routing for Text Classification , 2018, EMNLP.

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Michael Krauthammer,et al.  Term identification in the biomedical literature , 2004, J. Biomed. Informatics.

[5]  Alexander M. Rush,et al.  Structured Attention Networks , 2017, ICLR.

[6]  Zhiyong Lu,et al.  Benchmarking of the 2010 BioCreative Challenge III text-mining competition by the BioGRID and MINT interaction databases , 2011 .

[7]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[8]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[9]  Yijia Zhang,et al.  A hybrid model based on neural networks for biomedical relation extraction , 2018, J. Biomed. Informatics.

[10]  Adrian Tsang,et al.  Machine Learning for Biomedical Literature Triage , 2014, PloS one.

[11]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[12]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[13]  Zhiyong Lu,et al.  ML-Net: multi-label classification of biomedical texts with deep neural networks , 2018, J. Am. Medical Informatics Assoc..

[14]  Ramakanth Kavuluru,et al.  Convolutional neural networks for biomedical text classification: application in indexing biomedical articles , 2015, BCB.

[15]  William R. Hersh,et al.  Feature Generation, Feature Selection, Classifiers, and Conceptual Drift for Biomedical Document Triage , 2004, TREC.

[16]  Salman Khan,et al.  A Context-aware Capsule Network for Multi-label Classification , 2018, ECCV Workshops.

[17]  Jun Zhao,et al.  How to Generate a Good Word Embedding , 2015, IEEE Intelligent Systems.

[18]  Pushpak Bhattacharyya,et al.  A deep learning architecture for protein-protein Interaction Article identification , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[19]  Zhiyong Lu,et al.  Community challenges in biomedical text mining over 10 years: success, failure and the future , 2016, Briefings Bioinform..

[20]  F. Collins,et al.  A new initiative on precision medicine. , 2015, The New England journal of medicine.

[21]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[22]  Sunil Kumar Sahu,et al.  Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network , 2017, J. Biomed. Informatics.

[23]  Aaron M. Cohen,et al.  An Effective General Purpose Approach for Automated Biomedical Document Classification , 2006, AMIA.

[24]  A. Valencia,et al.  Overview of the protein-protein interaction annotation extraction task of BioCreative II , 2008, Genome Biology.

[25]  Qingyu Chen,et al.  Overview of the BioCreative VI Precision Medicine Track: mining protein interactions and mutations for precision medicine , 2019, Database J. Biol. Databases Curation.

[26]  Thomas C. Rindflesch,et al.  EDGAR: extraction of drugs, genes and relations from the biomedical literature. , 1999, Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing.

[27]  Luo Si,et al.  Thresholding Strategies for Text Classifiers: TREC 2005 Biomedical Triage Task Experiments , 2005, TREC.

[28]  Zhiyong Lu,et al.  Text Mining for Precision Medicine: Bringing Structure to EHRs and Biomedical Literature to Understand Genes and Health. , 2016, Advances in experimental medicine and biology.

[29]  Andrei Popescu-Belis,et al.  Multilingual Hierarchical Attention Networks for Document Classification , 2017, IJCNLP.