SemEval-2010 Task 1: Coreference Resolution in Multiple Languages

This paper presents the task "Coreference Resolution in Multiple Languages" to be run in SemEval-2010 (5th International Workshop on Semantic Evaluations). This task aims to evaluate and compare automatic coreference resolution systems for three different languages (Catalan, English, and Spanish) by means of two alternative evaluation metrics, thus providing an insight into (i) the portability of coreference resolution systems across languages, and (ii) the effect of different scoring metrics on ranking the output of the participant systems.

[1]  Lynette Hirschman,et al.  Appendix F: MUC-7 Coreference Task Definition (version 3.0) , 1998, MUC.

[2]  Maria Antònia Martí,et al.  AnCora-CO: Coreferentially annotated corpora for Spanish and Catalan , 2010, Lang. Resour. Evaluation.

[3]  Véronique Hoste,et al.  KNACK-2002: a Richly Annotated Corpus of Dutch Written Text , 2006, LREC.

[4]  Pascal Denis,et al.  Specialized Models and Ranking for Coreference Resolution , 2008, EMNLP.

[5]  Yannick Versley,et al.  Anaphoric Annotation of Wikipedia and Blogs in the Live Memories Corpus , 2010, LREC.

[6]  Mitchell P. Marcus,et al.  OntoNotes: A Unified Relational Semantic Representation , 2007, International Conference on Semantic Computing (ICSC 2007).

[7]  Constantin Orasan,et al.  Anaphora Resolution Exercise: an Overview , 2008, LREC.

[8]  PoesioMassimo,et al.  Two uses of anaphora resolution in summarization , 2007 .

[9]  Xiaoqiang Luo,et al.  On Coreference Resolution Performance Metrics , 2005, HLT.

[10]  Eduard Hovy,et al.  OntoNotes: A Unified Relational Semantic Representation , 2007 .

[11]  Helmut Schmid,et al.  Improvements in Part-of-Speech Tagging with an Application to German , 1999 .

[12]  Olga Uryupina,et al.  Corry: A System for Coreference Resolution , 2010, *SEMEVAL.

[13]  Hinrich Schütze,et al.  SUCRE: A Modular System for Coreference Resolution , 2010, *SEMEVAL.

[14]  Sandra Kübler,et al.  UBIU: A Language-Independent System for Coreference Resolution , 2010, *SEMEVAL.

[15]  Wendy G. Lehnert,et al.  Using Decision Trees for Coreference Resolution , 1995, IJCAI.

[16]  Erhard W. Hinrichs,et al.  A Unified Representation for Morphological, Syntactic, Semantic, and Referential Annotations , 2005, FCA@ACL.

[17]  Ruslan Mitkov,et al.  Robust Pronoun Resolution with Limited Knowledge , 1998, ACL.

[18]  Lluís Padró,et al.  RelaxCor: A Global Relaxation Labeling Approach to Coreference Resolution , 2010, ACL 2010.

[19]  Gertjan van Noord,et al.  Syntactic Annotation of Large Corpora in STEVIN , 2006, LREC.

[20]  Manuel Bertrán,et al.  AnCoraPipe: A tool for multilevel annotation , 2008, Proces. del Leng. Natural.

[21]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.

[22]  Yannick Versley,et al.  BART: A Multilingual Anaphora Resolution System , 2010, *SEMEVAL.

[23]  Helmut Schmid,et al.  Estimation of Conditional Probabilities With Decision Trees and an Application to Fine-Grained POS Tagging , 2008, COLING.

[24]  Thomas S. Morton,et al.  Using Coreference for Question Answering , 1999, TREC.

[25]  Karel Jezek,et al.  Two uses of anaphora resolution in summarization , 2007, Inf. Process. Manag..

[26]  Breck Baldwin,et al.  Algorithms for Scoring Coreference Chains , 1998 .

[27]  Maria Simi,et al.  TANL-1: Coreference Resolution by Parse Analysis and Similarity Clustering , 2010, *SEMEVAL.

[28]  Kees van Deemter,et al.  On Coreferring: Coreference in MUC and Related Annotation Schemes , 2000, CL.

[29]  Vincent Ng,et al.  Unsupervised Models for Coreference Resolution , 2008, EMNLP.

[30]  Lynette Hirschman,et al.  A Model-Theoretic Coreference Scoring Scheme , 1995, MUC.

[31]  M. R E C A S E,et al.  BLANC: Implementing the Rand index for coreference evaluation , 2010, Natural Language Engineering.

[32]  Joakim Nivre,et al.  A Dependency-Driven Parser for German Dependency and Constituency Representations , 2008, ACL 2008.

[33]  Eduard H. Hovy,et al.  BLANC: Implementing the Rand index for coreference evaluation , 2010, Natural Language Engineering.