论文信息 - Web-based Collaborative Corpus Annotation : Requirements and a Framework Implementation

Web-based Collaborative Corpus Annotation : Requirements and a Framework Implementation

In this paper we present Teamware, a novel web-based collaborative annotation environment which enables users to carry out complex corpus annotation projects, involving less skilled, cheaper annotators working remotely. It has been evaluated by us through the creation of several gold standard corpora, as well as through external evaluation in commercial annotation projects.

H. Cunningham | Kalina Bontcheva | Ian Roberts | V. Tablan

[1] Lynette Hirschman,et al. Mixed-Initiative Development of Language Processing Systems , 1997, ANLP.

[2] Elaine Marsh,et al. MUC-7 Evaluation of IE Technology: Overview of Results , 1998, MUC.

[3] David A. Ferrucci,et al. UIMA: an architectural approach to unstructured information processing in the corporate research environment , 2004, Natural Language Engineering.

[4] Laurel D. Riek,et al. Callisto: A Configurable Annotation Workbench , 2004, LREC.

[5] Stephanie Strassel,et al. Annotation Tools for Large-Scale Corpus Development: Using AGTK at the Linguistic Data Consortium , 2004, LREC.

[6] Philip V. Ogren,et al. Knowtator: A Protégé plug-in for annotated corpus construction , 2006, NAACL.

[7] Kalina Bontcheva,et al. Indexing and querying linguistic metadata and document content , 2007 .

[8] Brendan T. O'Connor,et al. Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.

[9] Kalina Bontcheva,et al. Large-scale, parallel automatic patent annotation , 2008, PaIR '08.

[10] Robert Parker,et al. Annotation Tool Development for Large-Scale Corpus Creation Projects at the Linguistic Data Consortium , 2008, LREC.

[11] Wim Peters,et al. SPRAT : a tool for automatic semantic pattern-based ontology population , 2009 .