论文信息 - Document-Level Automatic MT Evaluation based on Discourse Representations

Document-Level Automatic MT Evaluation based on Discourse Representations

This paper describes the joint submission of Universitat Politecnica de Catalunya and Universitat de Barcelona to the Metrics MaTr 2010 evaluation challenge, in collaboration with ELDA/ELRA. Our work is aimed at widening the scope of current automatic evaluation measures from sentence to document level. Preliminary experiments, based on an extension of the metrics by Gimenez and Marquez (2009) operating over discourse representations, are presented.

[1] Robert Tibshirani,et al. Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[2] Uwe Reyle,et al. From Discourse to Logic - Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory , 1993, Studies in linguistics and philosophy.

[3] Philipp Koehn,et al. Statistical Significance Tests for Machine Translation Evaluation , 2004, EMNLP.

[4] Ding Liu,et al. Syntactic Features for Evaluation of Machine Translation , 2005, IEEvaluation@ACL.

[5] Lluís Màrquez i Villodre,et al. Linguistic Features for Automatic Evaluation of Heterogenous MT Systems , 2007, WMT@ACL.

[6] Johan Bos,et al. Linguistically Motivated Large-Scale NLP with C&C and Boxer , 2007, ACL.

[7] Lluís Màrquez i Villodre,et al. A Smorgasbord of Features for Automatic MT Evaluation , 2008, WMT@ACL.

[8] Johan Bos,et al. Wide-Coverage Semantic Analysis with Boxer , 2008, STEP.

[9] Philipp Koehn,et al. Further Meta-Evaluation of Machine Translation , 2008, WMT@ACL.

[10] Philipp Koehn,et al. Findings of the 2009 Workshop on Statistical Machine Translation , 2009, WMT@EACL.

[11] Lluís Màrquez i Villodre,et al. On the Robustness of Syntactic and Semantic Features for Automatic MT Evaluation , 2009, WMT@EACL.