Negation and speculation scope detection using recursive neural conditional random fields

Abstract Negation and speculation scope detection is an important task in natural language processing. Previous studies all show that syntactic information is crucial to the task. However, these work mainly focuses on human-designed discrete features and local features extracted from dependency tree, limiting the performance of the task. In this paper, we propose a recursive neural network sequence labeling model, representing whole dependency tree globally and learning automatically syntactic features, for the task. Specifically, recursive neural network first learns a high-level representation for the words in the context of each sentence, and captures the cue word with its target scope through the global dependency structure. Then, CRFs layer takes the representation from recursive neural network as input to jointly decode labels for the whole sentence. Experimental results on English dataset BioScope and Chinese dataset CNeSp show that our model outperforms the state-of-the-art systems.

[1]  Sunghwan Mac Kim,et al.  Demographic Inference on Twitter using Recursive Neural Networks , 2017, ACL.

[2]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[3]  Bonnie L. Webber,et al.  Neural Networks for Cross-lingual Negation Scope Detection , 2018, ArXiv.

[4]  Roser Morante,et al.  *SEM 2012 Shared Task: Resolving the Scope and Focus of Negation , 2012, *SEMEVAL.

[5]  Changjin Xu,et al.  On anti-periodic solutions for neutral shunting inhibitory cellular neural networks with time-varying delays and D operator , 2018, Neurocomputing.

[6]  Changjin Xu,et al.  Local and global Hopf bifurcation analysis on simplified bidirectional associative memory neural networks with multiple delays , 2018, Math. Comput. Simul..

[7]  Roser Morante,et al.  Learning the Scope of Hedge Cues in Biomedical Texts , 2009, BioNLP@HLT-NAACL.

[8]  Yuji Matsumoto,et al.  Statistical Dependency Analysis with Support Vector Machines , 2003, IWPT.

[9]  Wendy W. Chapman,et al.  Evaluation of negation phrases in narrative clinical reports , 2001, AMIA.

[10]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[11]  Huaguang Zhang,et al.  Prediction and identification of discrete-time dynamic nonlinear systems based on adaptive echo state network , 2019, Neural Networks.

[12]  Donghong Ji,et al.  Long short-term memory RNN for biomedical named entity recognition , 2017, BMC Bioinformatics.

[13]  Guodong Zhou,et al.  Negation and Speculation Identification in Chinese Language , 2015, ACL.

[14]  Han Ren,et al.  Context-augmented convolutional neural networks for twitter sarcasm detection , 2018, Neurocomputing.

[15]  Guodong Zhou,et al.  Tree Kernel-based Negation and Speculation Scope Detection with Structured Syntactic Parse Features , 2013, EMNLP.

[16]  Xiaolong Wang,et al.  A Cascade Method for Detecting Hedges and their Scope in Natural Language Text , 2010, CoNLL Shared Task.

[17]  Changjin Xu,et al.  Global Exponential Convergence of Fuzzy Cellular Neural Networks with Leakage Delays, Distributed Delays and Proportional Delays , 2018, Circuits Syst. Signal Process..

[18]  Bonnie L. Webber,et al.  Neural Networks For Negation Scope Detection , 2016, ACL.

[19]  Yue Zhang,et al.  Context-Sensitive Twitter Sentiment Classification Using Neural Network , 2016, AAAI.

[20]  Richard Socher,et al.  A Neural Network for Factoid Question Answering over Paragraphs , 2014, EMNLP.

[21]  Xiaohui Liang,et al.  A hybrid neural network model for predicting kidney disease in hypertension patients based on electronic health records , 2019, BMC Medical Informatics and Decision Making.

[22]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[23]  Wei Luo,et al.  Speculation and Negation Scope Detection via Convolutional Neural Networks , 2016, EMNLP.

[24]  Stephan Oepen,et al.  Syntactic Scope Resolution in Uncertainty Analysis , 2010, COLING.

[25]  Changjin Xu,et al.  Global exponential convergence of neutral-type Hopfield neural networks with multi-proportional delays and leakage delays , 2017 .

[26]  Yafeng Ren,et al.  Detecting the Scope of Negation and Speculation in Biomedical Texts by Using Recursive Neural Network , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[27]  Virginia Francisco,et al.  UCM-2: a Rule-Based Approach to Infer the Scope of Negation via Dependency Parsing , 2012, *SEM@NAACL-HLT.

[28]  Roser Morante,et al.  Learning the Scope of Negation in Biomedical Texts , 2008, EMNLP.

[29]  Erik Velldal,et al.  UiO 2: Sequence-labeling Negation Using Dependency Features , 2012, *SEMEVAL.

[30]  Dragomir R. Radev,et al.  Detecting Speculations and their Scopes in Scientific Text , 2009, EMNLP.

[31]  János Csirik,et al.  The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes , 2008, BMC Bioinformatics.

[32]  Changjin Xu,et al.  Exponential Stability of Almost Periodic Solutions for Memristor-Based Neural Networks with Distributed Leakage Delays , 2016, Neural Computation.

[33]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[34]  Huaguang Zhang,et al.  Identification method for a class of periodic discrete-time dynamic nonlinear systems based on Sinusoidal ESN , 2018, Neurocomputing.

[35]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[36]  Quoc V. Le,et al.  Grounded Compositional Semantics for Finding and Describing Images with Sentences , 2014, TACL.

[37]  Stephan Oepen,et al.  Speculation and Negation: Rules, Rankers, and the Role of Syntax , 2012, CL.

[38]  Juan Martínez-Romo,et al.  Deep learning approach for negation trigger and scope recognition , 2019, Proces. del Leng. Natural.

[39]  Yang Huang,et al.  A novel hybrid approach to automated negation detection in clinical radiology reports. , 2007, Journal of the American Medical Informatics Association : JAMIA.

[40]  Noriko Tomuro,et al.  Automatic Extraction of Lexico-Syntactic Patterns for Detection of Negation and Speculation Scopes , 2011, ACL.

[41]  Xu Sun,et al.  Dependency-based Gated Recursive Neural Network for Chinese Word Segmentation , 2016, ACL.