Assessing the fidelity of consecutive interpreting

The study reported on in this article pertains to rater-mediated assessment of English-to-Chinese consecutive interpreting, particularly informational correspondence between an originally intended message and an actually rendered message, also known as “fidelity” in Interpreting Studies. Previous literature has documented two main methods to assess fidelity: comparing actual renditions with the source text or with an exemplar rendition carefully prepared by experts (i.e., an ideal target text). However, little is known about the potential effects of these methods on fidelity assessment. We therefore conducted the study to explore the way in which these methods would affect rater reliability, fidelity ratings and rater perception. Our analysis of quantitative data shows that the raters tended to be less reliable, less self-consistent, less lenient and less comfortable when using the source English text (i.e., Condition A) than when using the target Chinese text (i.e., Condition B: the exemplar rendition). These findings were backed up and explained by emerging themes derived from the qualitative questionnaire data. The fidelity estimates in the two conditions were also found to be strongly correlated. We discuss these findings and entertain the possibility of recruiting untrained monolinguals or bilinguals to assess fidelity of interpreting.

[1]  Tae-Hyung Lee Simultaneous listening and speaking in English into korean simultaneous interpretation , 1999 .

[2]  Sang-Bin Lee Developing an analytic scale for assessing undergraduate students’ consecutive interpreting performances , 2015 .

[3]  Minhua Liu,et al.  Assessing source material difficulty for consecutive interpreting: Quantifiable measures and holistic judgment , 2009 .

[4]  Daniel Gile,et al.  Fidelity assessment in consecutive interpretation : an experiment , 1995 .

[5]  John B. Carroll An experiment in evaluating the quality of translations , 1966, Mech. Transl. Comput. Linguistics.

[6]  Binhua Wang,et al.  Developing Interpreting Competence Scales in China , 2020, Frontiers in Psychology.

[7]  Robin Setton,et al.  Conference Interpreting. A trainer's guide , 2016 .

[8]  Chao Han Investigating rater severity/leniency in interpreter performance testing: A multifaceted Rasch measurement approach. , 2015 .

[9]  Chao Han Using analytic rating scales to assess English/Chinese bi-directional interpretation: A longitudinal Rasch analysis of scale utility and rater behavior , 2018, Linguistica Antverpiensia, New Series – Themes in Translation Studies.

[10]  Chao Han Latent trait modelling of rater accuracy in formative peer assessment of English-Chinese consecutive interpreting , 2018 .

[11]  Chao Han A generalizability theory study of optimal measurement design for a summative assessment of English/Chinese consecutive interpreting , 2018, Language Testing.

[12]  Edward W Wolfe,et al.  Detecting and measuring rater effects using many-facet Rasch measurement: part I. , 2003, Journal of applied measurement.

[13]  Sang-Bin Lee Holistic assessment of consecutive interpretation , 2019, Interpreting. International Journal of Research and Practice in Interpreting.

[14]  Robin Setton,et al.  Syntacrobatics: Quality and reformulation in simultaneous-with-text , 2007 .

[15]  Daniel Gile,et al.  Interpreting studies: a critical view from within , 2009 .

[16]  H. Barik Simultaneous Interpretation: Temporal and Quantitative Data , 1973, Language and speech.

[17]  Miriam Hamidi,et al.  Simultaneous Consecutive Interpreting: A New Technique Put to the Test , 2007 .

[18]  S. Campbell,et al.  Translation and interpreting assessment in the context of educational measurement , 2003 .

[19]  Chao Han Using rating scales to assess interpretation: Practices, problems and prospects , 2018 .

[20]  Diane L. Schallert,et al.  Working memory and expertise in simultaneous interpreting , 2004 .

[21]  Hildegund Bühler Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters , 1986 .

[22]  Claudia V. Angelelli,et al.  Testing and assessment in translation and interpreting studies : a call for dialogue between research and practice , 2009 .

[23]  D. Gile Variability in the perception of fidelity in simultaneous interpretation , 2017 .

[24]  Steven E. Stemler,et al.  Best Practices in Interrater Reliability Three Common Approaches , 2008 .

[25]  Chris Meuleman,et al.  Coping with extreme speech conditions in simultaneous interpreting , 2009 .

[26]  Chao Han,et al.  Investigating Score Dependability in English/Chinese Interpreter Certification Performance Testing: A Generalizability Theory Approach , 2016 .

[28]  Jim Hlavac,et al.  A Cross-National Overview of Translator and Interpreter Certification Procedures , 2013 .

[29]  Jieun Lee Rating Scales for Interpreting Performance Assessment , 2008 .