Translators’ perceptions of literary post-editing using statistical and neural machine translation

Abstract In the context of recent improvements in the quality of machine translation (MT) output and new use cases being found for that output, this article reports on an experiment using statistical and neural MT systems to translate literature. Six professional translators with experience of literary translation produced English-to-Catalan translations under three conditions: translation from scratch, neural MT post-editing, and statistical MT post-editing. They provided feedback before and after the translation via questionnaires and interviews. While all participants prefer to translate from scratch, mostly due to the freedom to be creative without the constraints of segment-level segmentation, those with less experience find the MT suggestions useful.

[1]  K. Reiss.  TYPE, KIND AND INDIVIDUALITY OF TEXT Decision Making in Translation * , 1981 .

[2]  Lucas Nunes Vieira.  Indices of cognitive effort in machine translation post-editing , 2014, Machine Translation.

[3]  Andy Way.  Machine translation: Where are we at today? , 2020 .

[4]  Yehoshua Bar-Hillel,et al.  The Present Status of Automatic Translation of Languages , 1960, Adv. Comput..

[5]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[6]  François Masselot,et al.  A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context , 2010, Prague Bull. Math. Linguistics.

[7]  Jeffrey Heer,et al.  The efficacy of human post-editing for language translation , 2013, CHI.

[8]  Kenneth Ward Church,et al.  Good applications for crummy machine translation , 1993, Machine Translation.

[9]  Kristiina Taivalkoski-Shilov,et al.  Ethical issues regarding machine(-assisted) translation of literary texts , 2018, Perspectives.

[10]  Lucia Specia,et al.  PET: a Tool for Post-editing and Assessing Machine Translation , 2012, LREC.

[11]  Hans P. Krings,et al.  Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes , 2001 .

[12]  M. Leblanc.  Translators on Translation Memory (TM). Results of an Ethnographic Study in Three Translation Services and Agencies , 2013 .

[13]  Harold L. Somers,et al.  Computers and translation : a translator's guide , 2003 .

[14]  Sharon O'Brien,et al.  Post-Editing Evaluations: Trade-offs between Novice and Professional Participants , 2015, EAMT.

[15]  Orphée De Clercq,et al.  Translationese and Post-editese: How comparable is comparable quality? , 2018, Linguistica Antverpiensia, New Series – Themes in Translation Studies.

[16]  Joss Moorkens,et al.  Under pressure: translation in times of austerity , 2017 .

[17]  Andy Way,et al.  What Level of Quality can Neural Machine Translation Attain on Literary Text? , 2018, ArXiv.

[18]  Andy Way,et al.  Post-editing Effort of a Novel With Statistical and Neural Machine Translation , 2018, Front. Digit. Humanit..

[19]  Andy Way,et al.  Quality expectations of machine translation , 2018, ArXiv.

[20]  Ann Irvine,et al.  The (Un)faithful Machine Translator , 2013, LaTeCH@ACL.

[21]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[22]  Mikel L. Forcada,et al.  Making sense of neural machine translation , 2017 .

[23]  Juan Alberto Alonso Martín,et al.  Integration of a Machine Translation System into the Editorial Process Flow of a Daily Newspaper , 2014, Proces. del Leng. Natural.

[24]  Sharon O'Brien,et al.  Analysing Post-Editing Performance: Correlations with Years of Translation Experience , 2010, EAMT.

[25]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[26]  Dragos Ciobanu,et al.  Traditional and Emerging Use-Cases for Machine Translation , 2013 .

[27]  Andy Way,et al.  Translating Literary Text between Related Languages using SMT , 2015, CLfL@NAACL-HLT.

[28]  E. Wagner,et al.  Post-editing SYSTRAN, a challenge for commission translators , 1985 .

[29]  Kevin Knight,et al.  Automatic Analysis of Rhythmic Poetry with Applications to Generation and Translation , 2010, EMNLP.

[30]  Ashish Vaswani,et al.  Decoding with Large-Scale Neural Language Models Improves Translation , 2013, EMNLP.

[31]  Andy Way,et al.  Is Neural Machine Translation the New State of the Art? , 2017, Prague Bull. Math. Linguistics.

[32]  Lijun Wu,et al.  Achieving Human Parity on Automatic Chinese to English News Translation , 2018, ArXiv.

[33]  Lucia Specia,et al.  Machine Translation Quality Estimation: Applications and Future Perspectives , 2018 .

[34]  Pascaline Dury,et al.  Lexicography, Terminology and Translation. Text-Based Studies in Honour of Ingrid Meyer , 2008 .

[35]  Nadir Durrani,et al.  A Joint Sequence Translation Model with Integrated Reordering , 2011, ACL.

[36]  Jennifer Pearson,et al.  Unity in Diversity : Current Trends in Translation Studies , 2016 .

[37]  Joss Moorkens,et al.  Chapter 4. Eye tracking as a measure of cognitive effort for post-editing of machine translation , 2018, Eye Tracking and Multidisciplinary Studies on Translation.

[38]  Matthew G. Snover,et al.  A Study of Translation Edit Rate with Targeted Human Annotation , 2006, AMTA.

[39]  Ana Guerberof Arenas.  Productivity and Quality in the Post-editing of Outputs from Translation Memories and Machine Translation , 2008 .

[40]  Arianna Bisazza,et al.  Neural versus phrase-based MT quality: An in-depth analysis on English-German and English-French , 2018, Comput. Speech Lang..

[41]  Linda Mitchell,et al.  Human factors in machine translation and post-editing among institutional translators , 2016 .

[42]  Michael Carl,et al.  Post-Editing Machine Translation: Efficiency, Strategies, and Revision Processes in Professional Translation Settings , 2015 .

[43]  Sharon O'Brien.  Post-editing of Machine Translation: Processes and Applications , 2014 .

[44]  Rico Sennrich,et al.  Nematus: a Toolkit for Neural Machine Translation , 2017, EACL.

[45]  Ralph Weischedel,et al.  A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[46]  Sharon O'Brien,et al.  Correlations of perceived post-editing effort with measurements of actual effort , 2015, Machine Translation.

[47]  Jakob Uszkoreit,et al.  “Poetic” Statistical Machine Translation: Rhyme and Meter , 2010, EMNLP.

[48]  Laurent Besacier.  Traduction automatisée d'une oeuvre littéraire: une étude pilote , 2014 .

[49]  Lucia Specia,et al.  Exploiting Objective Annotations for Minimising Translation Post-editing Effort , 2011, EAMT.

[50]  Carlos S. C. Teixeira.  Perceived vs. measured performance in the post-editing of suggestions from machine translation and translation memories , 2014, AMTA.

[51]  Anthony,et al.  ＂Professional Corpora＂： Teaching Strategies for Work with Online Documentation, Translation Memories and Content Management , 2008 .

[52]  J. C. Catford,et al.  A linguistic theory of translation : an essay in applied linguistics , 1965 .

[53]  Inna Kozlova,et al.  Investigating translation competence : conceptual and methodological issues , 2005 .

[54]  Philipp Koehn,et al.  Six Challenges for Neural Machine Translation , 2017, NMT@ACL.

[55]  Carlos S. C. Teixeira,et al.  Resistance and accommodation: factors for the (non-) adoption of machine translation among professional translators , 2017 .

[56]  Antonio Toral,et al.  A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions , 2017, EACL.

[57]  Antonio Toral,et al.  Fine-Grained Human Evaluation of Neural Versus Phrase-Based Machine Translation , 2017, Prague Bull. Math. Linguistics.

[58]  Federico Gaspari,et al.  Perception vs. reality: measuring machine translation post-editing productivity , 2014, AMTA.

[59]  Rico Sennrich,et al.  Controlling Politeness in Neural Machine Translation via Side Constraints , 2016, NAACL.

[60]  Andy Way,et al.  A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators , 2017, MTSUMMIT.

[61]  Rico Sennrich,et al.  Improving Neural Machine Translation Models with Monolingual Data , 2015, ACL.

[62]  Sheila Castilho,et al.  Content profiling and translation scenarios , 2016 .