Investigating the post-editing effort associated with machine-translated metaphors : a process-driven analysis

This paper reports on a study that analyses the impact of two different machine translation (MT) outputs on the cognitive effort required to post-edit machine-translated metaphors by means of eye tracking and think-aloud protocols. We hypothesise that the statistical MT output would have a positive effect on reducing cognitive effort. In order to test this hypothesis, a post-editing experiment was conducted with two different groups of participants. Each experimental group had two post-editing tasks using the language pair English into Brazilian Portuguese. On Task 1 (T1), participants were asked to postedit a Google machine-translated output whereas on Task 2 (T2) the same participants were assigned to post-edit a Systran machine translated output. Data collection was conducted under the experimental paradigm of data triangulation in translation process research. Data analysis focuses on eye tracking data related to fixation duration and pupil dilation as well as think-aloud protocols. This analysis shows that the cognitive effort required to post-edit the pure statistical MT output might be lower in comparison to the hybrid output when conventional metaphors are machine translated.

[1]  Marta R. Costa-jussà,et al.  Study and Comparison of Rule-Based and Statistical Catalan-Spanish Machine Translation Systems , 2012, Comput. Informatics.

[2]  Liu Qun,et al.  Machine translation: general , 2014 .

[3]  L. Barsalou,et al.  Ad hoc categories , 1983, Memory & cognition.

[5]  E. Romero,et al.  RELEVANCE THEORY AND METAPHOR , 2014 .

[6]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[7]  Michael Carl,et al.  The Process of Post-Editing: A Pilot Study , 2011 .

[8]  Gibbs,et al.  Figurative thought and figurative language. , 1994 .

[9]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[10]  R. Gibbs,et al.  Complementary perspectives on metaphor: Cognitive linguistics and relevance theory. , 2008 .

[11]  Raymond W. Gibbs,et al.  Cognitive Effort and Effects in Metaphor Comprehension: Relevance Theory and Psycholinguistics , 2006 .

[12]  Philipp Koehn,et al.  Neural Machine Translation , 2017, ArXiv.

[13]  Philipp Koehn,et al.  Statistical Post-Editing on SYSTRAN‘s Rule-Based Translation System , 2007, WMT@ACL.

[14]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[15]  Sharon O'Brien,et al.  Correlations of perceived post-editing effort with measurements of actual effort , 2015, Machine Translation.

[16]  S Sreelekha,et al.  Statistical Vs Rule Based Machine Translation; A Case Study on Indian Language Perspective , 2017, ArXiv.

[17]  Robert A. Harris,et al.  Figurative language , 2021, The Craft of Poetry.

[18]  Robyn Carston,et al.  Thoughts and Utterances: The Pragmatics of Explicit Communication , 2002 .