and Machine Learnability in Computer Aided Translation

Analysesofcomputeraidedtranslationtypically focus on either frontend interfaces and human eort, or backend translation and machine learnability of corrections. However, this distinction is artificial in practice since the frontend and backend must work in concert. We present the first holistic, quantitative evaluation of these issues by contrasting two assistive modes: postediting and interactive machine translation (MT). We describe a new translator interface, extensive modifications to a phrasebased MT system, and a novel objective function for re-tuning to human corrections. Evaluation with professional bilingualtranslatorsshowsthatpost-editisfaster than interactive at the cost of translation quality for French-English and EnglishGerman. However, re-tuning the MT system to interactive output leads to larger, statistically significant reductions in HTER versus re-tuning to post-edit. Analysis shows that tuning directly to HTER results in fine-grained corrections to subsequent machine output.

[1]  Thorsten Joachims,et al.  Online Structured Prediction via Coactive Learning , 2012, ICML.

[2]  Philipp Koehn,et al.  Scalable Modified Kneser-Ney Language Model Estimation , 2013, ACL.

[3]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[4]  Philip Koehn,et al.  Statistical Machine Translation , 2010, EAMT.

[5]  Alon Lavie,et al.  Real Time Adaptive Machine Translation for Post-Editing with cdec and TransCenter , 2014, HaCaT@EACL.

[6]  George F. Foster,et al.  Batch Tuning Strategies for Statistical Machine Translation , 2012, NAACL.

[7]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[8]  Philip Resnik,et al.  Evaluating Translational Correspondence using Annotation Projection , 2002, ACL.

[9]  Hermann Ney,et al.  Statistical Approaches to Computer-Assisted Translation , 2009, CL.

[10]  Francisco Casacuberta,et al.  Advanced computer aided translation with a web-based workbench , 2013, MTSUMMIT.

[11]  Jeffrey Heer,et al.  Predictive translation memory: a mixed-initiative system for human language translation , 2014, UIST.

[12]  Francisco Casacuberta,et al.  The New Thot Toolkit for Fully-Automatic and Interactive Statistical Machine Translation , 2014, EACL.

[13]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[14]  Stefan Riezler,et al.  On Some Pitfalls in Automatic Evaluation and Significance Testing for MT , 2005, IEEvaluation@ACL.

[15]  Francisco Casacuberta,et al.  Online Learning for Interactive Statistical Machine Translation , 2010, NAACL.

[16]  Philipp Koehn,et al.  Findings of the 2014 Workshop on Statistical Machine Translation , 2014, WMT@ACL.

[17]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[18]  Ben Taskar,et al.  Alignment by Agreement , 2006, NAACL.

[19]  Philipp Koehn,et al.  Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[20]  Marcello Federico,et al.  Online Word Alignment for Online Adaptive Machine Translation , 2014, HaCaT@EACL.

[21]  Hermann Ney,et al.  Efficient Search for Interactive Statistical Machine Translation , 2003, EACL.

[22]  Hermann Ney,et al.  The Alignment Template Approach to Statistical Machine Translation , 2004, CL.

[23]  Nitin Madnani,et al.  Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric , 2009, WMT@EACL.

[24]  David Chiang,et al.  Forest Rescoring: Faster Decoding with Integrated Language Models , 2007, ACL.

[25]  Pascual Martínez-Gómez,et al.  Online adaptation strategies for statistical machine translation in post-editing scenarios , 2012, Pattern Recognit..

[26]  George F. Foster,et al.  TransType: a Computer-Aided Translation Typing System , 2000 .

[27]  Ralph Weischedel,et al.  A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[28]  Christopher D. Manning,et al.  Phrasal: A Toolkit for New Directions in Statistical Machine Translation , 2014, WMT@ACL.

[29]  Alon Lavie,et al.  Extending the METEOR Machine Translation Evaluation Metric to the Phrase Level , 2010, NAACL.

[30]  Philipp Koehn,et al.  Findings of the 2013 Workshop on Statistical Machine Translation , 2013, WMT@ACL.

[31]  Philipp Koehn,et al.  Re-evaluating the Role of Bleu in Machine Translation Research , 2006, EACL.

[32]  Francisco Casacuberta,et al.  Interactive Machine Translation Based on Partial Statistical Phrase-based Alignments , 2009, RANLP.

[33]  Yehoshua Bar-Hillel,et al.  The Present Status of Automatic Translation of Languages , 1960, Adv. Comput..

[34]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[35]  Matt Post,et al.  Efficient Elicitation of Annotations for Human Evaluation of Machine Translation , 2014, WMT@ACL.

[36]  Alon Lavie,et al.  Learning from Post-Editing: Online Model Adaptation for Statistical Machine Translation , 2014, EACL.

[37]  Philipp Koehn,et al.  A Web-Based Interactive Computer Aided Translation Tool , 2009, ACL.