The Impact of Machine Translation Quality on Human Post-Editing

We investigate the effect of four different competitive machine translation systems on post-editor productivity and behaviour. The study involves four volunteers postediting automatic translations of news stories from English to German. We see significant difference in productivity due to the systems (about 20%), and even bigger variance between post-editors.

[1]  Jörg Tiedemann,et al.  Tunable Distortion Limits and Corpus Cleaning for SMT , 2013, WMT@ACL.

[2]  Jan Niehues,et al.  The IWSLT 2011 Evaluation Campaign on Automatic Talk Translation , 2012, LREC.

[3]  Maarit Koponen,et al.  Comparing human perceptions of post-editing effort with post-editing operations , 2012, WMT@NAACL-HLT.

[4]  Philipp Koehn,et al.  Enabling Monolingual Translators: Post-Editing vs. Options , 2010, NAACL.

[5]  K. Glasgow,et al.  Los Angeles, California , 2003 .

[6]  Martin Volk,et al.  Assessing post-editing efficiency in a realistic translation environment , 2013, MTSUMMIT.

[7]  Ana Guerberof Arenas Productivity and Quality in MT Post-editing , 2009, MTSUMMIT.

[8]  Alex Waibel,et al.  Proceedings of the International Workshop on Spoken Language Translation (IWSLT 2013) , 2010 .

[9]  Lucia Morado Vazquez,et al.  Comparing forum data post-editing performance using translation memory and machine translation output: a pilot study , 2013 .

[10]  Jeffrey Heer,et al.  The efficacy of human post-editing for language translation , 2013, CHI.

[11]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[12]  Ignacio Garcia,et al.  Translating by post-editing: is it the way forward? , 2011, Machine Translation.

[13]  Nadir Durrani,et al.  Edinburgh’s Machine Translation Systems for European Language Pairs , 2013, WMT@ACL.

[14]  Philipp Koehn,et al.  Edinburgh’s Syntax-Based Machine Translation Systems , 2013, WMT@ACL.

[15]  Philipp Koehn,et al.  A process study of computer-aided translation , 2009, Machine Translation.

[16]  Philipp Koehn,et al.  Findings of the 2013 Workshop on Statistical Machine Translation , 2013, WMT@ACL.

[17]  Philipp Koehn,et al.  Findings of the 2012 Workshop on Statistical Machine Translation , 2012, WMT@NAACL-HLT.

[18]  François Masselot,et al.  A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context , 2010, Prague Bull. Math. Linguistics.

[19]  Ralph Weischedel,et al.  A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[20]  Lucia Specia,et al.  Post-editing time as a measure of cognitive effort , 2012, AMTA.