A Hybrid Deep Learning Framework for Bacterial Named Entity Recognition

Microorganisms have been confirmed to be essential for the fundamental function of various ecosystems. The interactions among microorganisms affect the human health and environmental ecosystem. A large number of microbial interactions with experimental confidence have been reported in biomedical literature. Extracting and collating these interactions with experimental confidence into a database will create a valuable data resource. Named Entity Recognition (NER) is the premise and key to interaction extraction from literatures. Especially, bacterial named entity recognition is still a challenging task due to the specialty of bacterial names. In this paper, we propose a bacterial named entity recognition system based on a hybrid deep learning framework (HDL-CRF), which integrates two deep learning models: the bidirectional long short-term memory network and the convolutional neural network, as well as the conditional random field approach, for automatically extracting the features. Finally, we prove that this model outperforms previous methods in performance.

[1]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[2]  Roded Sharan,et al.  The large-scale organization of the bacterial network of ecological co-occurrence interactions , 2010, Nucleic acids research.

[3]  Chengjie Sun,et al.  LSTM-CRF for Drug-Named Entity Recognition , 2017, Entropy.

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Manish Kumar,et al.  Recent Named Entity Recognition and Classification techniques: A systematic review , 2018, Comput. Sci. Rev..

[6]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[7]  Niranjan Nagarajan,et al.  Predicting microbial interactions through computational approaches. , 2016, Methods.

[8]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[9]  Rob Knight,et al.  Longitudinal analysis of microbial interaction between humans and the indoor environment , 2014, Science.

[10]  Niranjan Nagarajan,et al.  @MInter: automated text-mining of microbial interactions , 2016, Bioinform..

[11]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[12]  Hatem Haddad,et al.  Arabic Named Entity Recognition: A Bidirectional GRU-CRF Approach , 2017, CICLing.

[13]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[16]  T A Anderson,et al.  Microbial degradation of trichloroethylene in the rhizosphere: potential application to biological remediation of waste sites , 1990, Applied and environmental microbiology.

[17]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[18]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[19]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[20]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[21]  William R. Hersh,et al.  A survey of current work in biomedical text mining , 2005, Briefings Bioinform..

[22]  Xiaoyan Wang,et al.  Bacterial named entity recognition based on dictionary and conditional random field , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[23]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[24]  Xingpeng Jiang,et al.  Recognition of bacteria named entity using conditional random fields in Spark , 2018, BMC Systems Biology.

[25]  S. Chisholm,et al.  Host-Microbe Interactions: Shaping the Evolution of the Plant Immune Response , 2022 .

[26]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[27]  Christoph Goller,et al.  Learning task-dependent distributed representations by backpropagation through structure , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[28]  Wang Ling,et al.  Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation , 2015, EMNLP.

[29]  Sampo Pyysalo,et al.  How to Train good Word Embeddings for Biomedical NLP , 2016, BioNLP@ACL.

[30]  Forest Rohwer,et al.  Metagenomic analysis of the microbial community associated with the coral Porites astreoides. , 2007, Environmental microbiology.

[31]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[32]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .