Biological Event Trigger Identification with Noise Contrastive Estimation

Biological Event Extraction is an important task towards the goal of extracting biomedical knowledge from the scientific publications by capturing biomedical entities and their complex relations from the texts. As a crucial step in event extraction, event trigger identification, assigning words with suitable trigger category, has recently attracted substantial attention. As triggers are scattered in large corpus, traditional linguistic parsers are hard to generate syntactic features from them. Thereby, trigger sparsity problem restricts the model's learning process and becomes one of the main hinder in trigger identification. In this paper, we employ Noise Contrastive Estimation with Multi-Layer Perceptron model for solving triggers’ sparsity problem. Meanwhile, in the light of recent advance in word distributed representation, word-embedding feature generated by language model is utilized for semantic and syntactic information extraction. Finally, experimental study on commonly used MLEE dataset against baseline methods has demonstrated its promising result.

[1]  Sampo Pyysalo,et al.  Event extraction across multiple levels of biological organization , 2012, Bioinform..

[2]  Jun'ichi Tsujii,et al.  Event Extraction from Biomedical Papers Using a Full Parser , 2000, Pacific Symposium on Biocomputing.

[3]  Sophia Ananiadou,et al.  NaCTeM EventMine for BioNLP 2013 CG and PC tasks , 2013, BioNLP@ACL.

[4]  K. Bretonnel Cohen,et al.  HIGH‐PRECISION BIOLOGICAL EVENT EXTRACTION: EFFECTS OF SYSTEM AND OF DATA , 2011, Comput. Intell..

[5]  Jianlin Cheng,et al.  A Deep Learning Network Approach to ab initio Protein Secondary Structure Prediction , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[6]  Jari Björne,et al.  Generalizing Biomedical Event Extraction , 2011, BioNLP@ACL.

[7]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[8]  Stephanie Seneff,et al.  Using word embedding for bio-event extraction , 2015, BioNLP@IJCNLP.

[9]  Sophia Ananiadou,et al.  What causes a causal relation? Detecting Causal Triggers in Biomedical Scientific Discourse , 2013, ACL.

[10]  Jari Björne,et al.  Complex event extraction at PubMed scale , 2010, Bioinform..

[11]  Yee Whye Teh,et al.  A fast and simple algorithm for training neural probabilistic language models , 2012, ICML.

[12]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[13]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[14]  Junwu Zhu,et al.  Empirical studies on the NLP techniques for source code data preprocessing , 2014, EAST 2014.

[15]  Aapo Hyvärinen,et al.  Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics , 2012, J. Mach. Learn. Res..

[16]  Doheon Lee,et al.  Predicting the Absorption Potential of Chemical Compounds Through a Deep Learning Approach , 2018, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[17]  Jun'ichi Tsujii,et al.  Dependency Parsing and Domain Adaptation with LR Models and Parser Ensembles , 2007, EMNLP.

[18]  Vibhu O. Mittal,et al.  Stemming and its effects on TFIDF ranking. , 2000, SIGIR 2000.

[19]  Zhang Xiong,et al.  Embedding assisted prediction architecture for event trigger identification , 2015, J. Bioinform. Comput. Biol..

[20]  Zhenchao Jiang,et al.  An Unsupervised Graph Based Continuous Word Representation Method for Biomedical Text Mining , 2016, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[21]  Udo Hahn,et al.  Evaluating the Impact of Alternative Dependency Graph Encodings on Solving Event Extraction Tasks , 2010, EMNLP.

[22]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[23]  Edward Y. Chang,et al.  Class-Boundary Alignment for Imbalanced Dataset Learning , 2003 .

[24]  Anderson Rocha,et al.  Multiclass From Binary: Expanding One-Versus-All, One-Versus-One and ECOC-Based Approaches , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Jun Zhao,et al.  How to Generate a Good Word Embedding , 2015, IEEE Intelligent Systems.

[26]  Honglei Li,et al.  DUTIR in BioNLP-ST 2016: Utilizing Convolutional Network and Distributed Representation to Extract Complicate Relations , 2016, BioNLP.

[27]  Alberto Lavelli,et al.  Impact of Less Skewed Distributions on Efficiency and Effectiveness of Biomedical Relation Extraction , 2012, COLING.

[28]  Hui Chen,et al.  Chemical named entity recognition in patents by domain knowledge and unsupervised feature learning , 2016, Database J. Biol. Databases Curation.

[29]  György Móra,et al.  Exploring ways beyond the simple supervised learning approach for biological event extraction , 2009, BioNLP@HLT-NAACL.

[30]  Mark Sanderson,et al.  Word sense disambiguation and information retrieval , 1994, SIGIR '94.

[31]  Sampo Pyysalo,et al.  Overview of BioNLP Shared Task 2013 , 2013, BioNLP@ACL.

[32]  Sampo Pyysalo,et al.  Overview of the ID, EPI and REL tasks of BioNLP Shared Task 2011 , 2012, BMC Bioinformatics.

[33]  Ralph Grishman,et al.  Semi-supervised Relation Extraction with Large-scale Word Clustering , 2011, ACL.

[34]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[35]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[36]  Koray Kavukcuoglu,et al.  Learning word embeddings efficiently with noise-contrastive estimation , 2013, NIPS.

[37]  M. Neeman,et al.  The antiangiogenic agent linomide inhibits the growth rate of von Hippel-Lindau paraganglioma xenografts to mice. , 1999, Clinical cancer research : an official journal of the American Association for Cancer Research.

[38]  Yang Li,et al.  Mining evidences for named entity disambiguation , 2013, KDD.

[39]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[40]  Zhiyong Lu,et al.  Community challenges in biomedical text mining over 10 years: success, failure and the future , 2016, Briefings Bioinform..

[41]  Deyu Zhou,et al.  Event trigger identification for biomedical events extraction using domain knowledge , 2014, Bioinform..

[42]  D. E. Dimla,et al.  On-line metal cutting tool condition monitoring.: II: tool-state classification using multi-layer perceptron neural networks , 2000 .

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Shanshan Liu,et al.  Extracting Biomedical Event with Dual Decomposition Integrating Word Embeddings , 2016, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[45]  Casey S. Greene,et al.  Recent Advances and Emerging Applications in Text and Data Mining for Biomedical Discovery , 2015, Briefings Bioinform..

[46]  Jun'ichi Tsujii,et al.  Task-oriented Evaluation of Syntactic Parsers and Their Representations , 2008, ACL.

[47]  Yiyu Yao,et al.  Micro and macro evaluation of classification rules , 2008, 2008 7th IEEE International Conference on Cognitive Informatics.

[48]  Sampo Pyysalo,et al.  Overview of BioNLP’09 Shared Task on Event Extraction , 2009, BioNLP@HLT-NAACL.