A Graphical Interface for MT Evaluation and Error Analysis

Error analysis in machine translation is a necessary step in order to investigate the strengths and weaknesses of the MT systems under development and allow fair comparisons among them. This work presents an application that shows how a set of heterogeneous automatic metrics can be used to evaluate a test bed of automatic translations. To do so, we have set up an online graphical interface for the Asiya toolkit, a rich repository of evaluation measures working at different linguistic levels. The current implementation of the interface shows constituency and dependency trees as well as shallow syntactic and semantic annotations, and word alignments. The intelligent visualization of the linguistic structures used by the metrics, as well as a set of navigational functionalities, may lead towards advanced methods for automatic error analysis.

[1]  Hermann Ney,et al.  Towards Automatic Error Analysis of Machine Translation Output , 2011, CL.

[2]  Maja Popovic.  Hjerson: An Open Source Tool for Automatic Error Classification of Machine Translation Output , 2011, Prague Bull. Math. Linguistics.

[3]  Nizar Habash,et al.  Automatic Error Analysis for Morphologically Rich Languages , 2011 .

[4]  Lluís Màrquez i Villodre,et al.  Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation , 2010, Prague Bull. Math. Linguistics.

[5]  Nizar Habash,et al.  Semi-automatic error analysis for large-scale statistical machine translation , 2007, MTSUMMIT.

[6]  Ondrej Bojar,et al.  Automatic Translation Error Analysis , 2011, TSD.

[7]  José B. Mariño,et al.  Overcoming statistical machine translation limitations: error analysis and proposed solutions for the Catalan–Spanish language pair , 2011, Lang. Resour. Evaluation.

[8]  Lluís Màrquez i Villodre,et al.  Towards Heterogeneous Automatic MT Error Analysis , 2008, LREC.

[9]  Lluís Màrquez i Villodre,et al.  Linguistic measures for automatic machine translation evaluation , 2010, Machine Translation.

[10]  Julio Gonzalo,et al.  Corroborating Text Evaluation Results with Heterogeneous Measures , 2011, EMNLP.

[11]  José B. Mariño,et al.  Morpho-syntactic Information for Automatic Error Analysis of Statistical Machine Translation Output , 2006, WMT@HLT-NAACL.

[12]  Sara Stymne,et al.  Blast: A Tool for Error Analysis of Machine Translation Output , 2011, ACL.

[13]  Hermann Ney,et al.  Error Analysis of Statistical Machine Translation Output , 2006, LREC.