论文信息 - Using Derivation Trees for Informative Treebank Inter-Annotator Agreement Evaluation

Using Derivation Trees for Informative Treebank Inter-Annotator Agreement Evaluation

This paper discusses the extension of a system developed for automatic discovery of treebank annotation inconsistencies over an entire corpus to the particular case of evaluation of inter-annotator agreement. This system makes for a more informative IAA evaluation than other systems because it pinpoints the inconsistencies and groups them by their structural types. We evaluate the system on two corpora - (1) a corpus of English web text, and (2) a corpus of Modern British English.

[1] Hinrich Schütze,et al. Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[2] Seth Kulick,et al. Further Developments in Treebank Error Detection Using Derivation Trees , 2012, LREC.

[3] David Chiang,et al. Statistical Parsing with an Automatically-Extracted Tree Adjoining Grammar , 2000, ACL.

[4] Walt Detmar Meurers,et al. Detecting Inconsistencies in Treebanks , 2003 .

[5] Yoav Goldberg,et al. Inspecting the Structural Biases of Dependency Parsing Algorithms , 2010, CoNLL.

[6] Seth Kulick,et al. Using Derivation Trees for Treebank Error Detection , 2011, ACL.

[7] Libin Shen,et al. A new resource for incremental, dependency and semantic parsing , 2008 .

[8] Aravind K. Joshi,et al. A Formal Look at Dependency Grammars and Phrase-Structure Grammars, with Special Consideration of Word-Order Phenomena , 1994, ArXiv.

[9] Markus Dickinson. Detecting Errors in Automatically-Parsed Dependency Relations , 2010, ACL.

[10] Walt Detmar Meurers,et al. Increasing the Recall of Corpus Annotation Error Detection , 2007 .

[11] Aravind K. Joshi,et al. Tree-Adjoining Grammars , 1997, Handbook of Formal Languages.