The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text

The CoNLL-2010 Shared Task was dedicated to the detection of uncertainty cues and their linguistic scope in natural language texts. The motivation behind this task was that distinguishing factual and uncertain information in texts is of essential importance in information extraction. This paper provides a general overview of the shared task, including the annotation protocols of the training and evaluation datasets, the exact task definitions, the evaluation metrics employed and the overall results. The paper concludes with an analysis of the prominent approaches and an overview of the systems submitted to the shared task.

[1]  Carol Friedman,et al.  Research Paper: A General Natural-language Text Processor for Clinical Radiology , 1994, J. Am. Medical Informatics Assoc..

[2]  G. Lakoff.  Linguistics and natural logic , 1970, Synthese.

[3]  Padmini Srinivasan,et al.  The Language of Bioscience: Facts, Speculations, and Statements In Between , 2004, HLT-NAACL 2004.

[4]  Wendy W. Chapman,et al.  ConText: An Algorithm for Identifying Contextual Features from Clinical Text , 2007, BioNLP@ACL.

[5]  Ted Briscoe,et al.  Weakly Supervised Learning for Hedge Classification in Scientific Literature , 2007, ACL.

[6]  János Csirik,et al.  The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes , 2008, BMC Bioinformatics.

[7]  Halil Kilicoglu,et al.  Recognizing speculative language in biomedical research articles: a linguistically motivated perspective , 2008, BMC Bioinformatics.

[8]  György Szarvas,et al.  Hedge Classification in Biomedical Texts with a Weakly Supervised Selection of Keywords , 2008, ACL.

[9]  Son Doan,et al.  Using Hedges to Enhance a Disease Outbreak Report Text Mining System , 2009, BioNLP@HLT-NAACL.

[10]  Kazuhiko Ohe,et al.  TEXT2TABLE: Medical Text Summarization System Based on Named Entity Recognition and Modality Identification , 2009, BioNLP@HLT-NAACL.

[11]  Yvan Saeys,et al.  Analyzing text in search of bio-molecular events: a high-precision machine learning framework , 2009, BioNLP@HLT-NAACL.

[12]  Dragomir R. Radev,et al.  Detecting Speculations and their Scopes in Scientific Text , 2009, EMNLP.

[13]  Halil Kilicoglu,et al.  Syntactic Dependency Based Heuristics for Biological Event Extraction , 2009, BioNLP@HLT-NAACL.

[14]  Michael Strube,et al.  Finding Hedges by Chasing Weasels: Hedge Detection Using Wikipedia Tags and Shallow Linguistic Features , 2009, ACL.

[15]  K. Bretonnel Cohen,et al.  U-Compare: share and compare text mining tools with UIMA , 2009, Bioinform..

[16]  Sampo Pyysalo,et al.  Overview of BioNLP’09 Shared Task on Event Extraction , 2009, BioNLP@HLT-NAACL.

[17]  Roser Morante,et al.  Learning the Scope of Hedge Cues in Biomedical Texts , 2009, BioNLP@HLT-NAACL.

[18]  Xuanjing Huang,et al.  Detecting Hedge Cues and their Scopes with Average Perceptron , 2010, CoNLL Shared Task.

[19]  Hai Zhao,et al.  Hedge Detection and Scope Finding by Sequence Labeling with Procedural Feature Selection , 2010, CoNLL Shared Task.

[20]  Roser Morante,et al.  Memory-Based Resolution of In-Sentence Scopes of Hedge Cues , 2010, CoNLL Shared Task.

[21]  Xiaolong Wang,et al.  A Cascade Method for Detecting Hedges and their Scope in Natural Language Text , 2010, CoNLL Shared Task.

[22]  Xiaoyan Li,et al.  Exploiting Multi-Features to Detect Hedges and their Scope in Biomedical Texts , 2010, CoNLL Shared Task.

[23]  Qi Zhao,et al.  Learning to Detect Hedges and their Scope Using CRF , 2010, CoNLL Shared Task.

[24]  Halil Kilicoglu,et al.  A High-Precision Approach to Detecting Hedges and their Scopes , 2010, CoNLL Shared Task.

[25]  Guodong Zhou,et al.  Hedge detection and scope finding by sequence labeling with normalized feature selection , 2010, CoNLL 2010.

[26]  Xuan Wang,et al.  Exploiting Rich Features for Detecting Hedges and their Scope , 2010, CoNLL Shared Task.