Measuring translation difficulty: Theoretical and methodological considerations

Accurate assessment of a text’s level of translation difficulty is critical for translator training, accreditation and research. Traditionally, people rely on their general impression to gauge a text’s translation difficulty level. If the evaluation process is to be more effective and the results more objective, an instrument needs to be developed. Then two basic research questions must be answered: what to measure and how to measure it. The potential sources of translation difficulty include translation factors (i.e., text difficulty and translation-specific difficulty) and translator factors. Accordingly, to measure translation difficulty, we need to measure text difficulty, identify translation-specific difficulty, and assess translation difficulty (i.e., mental workload) for the translator. Readability formulas are often used to measure text difficulty. The means for identifying translation-specific difficulty include grading translations, analyzing verbal protocols, and recording and analyzing transl...

[1]  Keiran J. Dunne Assessing software localization: For a valid approach , 2009 .

[2]  Robert L. Johnson,et al.  Assessing Performance: Designing, Scoring, and Validating Performance Tasks , 2008 .

[3]  Mona Baker,et al.  In Other Words: A Coursebook on Translation , 1993 .

[4]  Kristian Tangsgaard Hvelplund Jensen,et al.  Indicators of Text Complexity , 2009 .

[5]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[6]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[7]  Janet Fraser The Translator Investigated , 1996 .

[8]  Esther Usó-Juan The Compensatory Nature of Discipline-Related Knowledge and English-Language Proficiency in Reading English for Academic Purposes , 2006 .

[9]  Joni M. Lakin,et al.  The Cambridge Handbook of Intelligence: Intelligence and Reasoning , 2011 .

[10]  N. Moray Mental workload : its theory and measurement , 1979 .

[11]  Slava Kalyuga,et al.  Managing Cognitive Load in Adaptive Multimedia Learning , 2008 .

[12]  F. Paas Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. , 1992 .

[13]  Wilson L. Taylor,et al.  “Cloze Procedure”: A New Tool for Measuring Readability , 1953 .

[14]  J. Chall,et al.  Readability revisited : the new Dale-Chall readability formula , 1995 .

[15]  F. Thomas Eggemeier,et al.  Workload assessment methodology. , 1986 .

[16]  Arnt Lykke Jakobsen,et al.  Tracking translators’ keystrokes and eye movements with Translog , 2011 .

[17]  Anthony Pym,et al.  Exploring Translation Theories , 2009 .

[18]  Craig A. Kaplan,et al.  Foundations of cognitive science , 1989 .

[19]  Roland Brünken,et al.  Cognitive Load Theory: THEORY , 2010 .

[20]  Mary Snell-Hornby,et al.  Translation Studies: An integrated approach , 1988 .

[21]  Christina Schäffner,et al.  Developing translation competence , 2000 .

[22]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[23]  David Y. W. Lee,et al.  Genres, Registers, Text Types, Domains and Styles: Clarifying the Concepts and Navigating a Path through the BNC Jungle , 2001 .

[24]  Brian C. McKevitt,et al.  Best Practices in School Psychology IV , 1990 .

[25]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[26]  E. Fry,et al.  Reading Formulas--Maligned but Valid. , 1989 .

[27]  Mona Baker,et al.  Routledge Encyclopedia of Translation Studies , 2009 .

[28]  Michael A. Vidulich,et al.  The Cognitive Psychology of Subjective Mental Workload , 1988 .

[29]  Jeanne S. Chall,et al.  Qualitative Assessment of Text Difficulty. A Practical Guide for Teachers and Writers. Working Paper Series. , 1996 .

[30]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[31]  L. Kaufman,et al.  Handbook of Perception and Human Performance. Volume 2. Cognitive Processes and Performance , 1994 .

[32]  J. Bradshaw,et al.  Load and pupillary changes in continuous processing tasks. , 1968, British journal of psychology.

[33]  G. Zipf The Psycho-Biology Of Language: AN INTRODUCTION TO DYNAMIC PHILOLOGY , 1999 .

[34]  Birgitta Englund Dimitrova Expertise and explicitation in the translation process , 2005 .

[35]  Henry R. Jex,et al.  Measuring Mental Workload: Problems, Progress, and Promises , 1988 .

[36]  Daniel Gopher,et al.  Workload: An examination of the concept. , 1986 .

[37]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[38]  P. David Pearson Handbook of reading research. , 1990 .

[39]  Christiane Nord,et al.  Text analysis in translation : theory, methodology, and didactic application of a model for translation-oriented text analysis. , 1992 .

[40]  Edward E. Gickling,et al.  Best Practices in Curriculum-Based Assessment. , 2002 .

[41]  M. Chi Two Approaches to the Study of Experts' Characteristics , 2006 .

[42]  Waldemar Karwowski,et al.  Mental Workload Measurement , 2006 .

[43]  Sanjun Sun Think-Aloud-Based Translation Process Research: Some Methodological Considerations , 2012 .

[44]  D. Biber A typology of English texts , 1989 .

[45]  George R. S. Weir,et al.  From corpus-based collocation frequencies to readability measure , 2006 .

[46]  Peter Robinson,et al.  Second language task complexity : researching the cognition hypothesis of language learning and performance , 2011 .

[47]  Pacte Building a translation competence model , 2003 .

[48]  William H. DuBay The Principles of Readability. , 2004 .

[49]  Corinna E. Lathan,et al.  Simulation Fidelity Design Informed by Physiologically-Based Measurement Tools , 2007, HCI.

[50]  Matthew K. Burns,et al.  Empirical Analysis of Drill Ratio Research , 2004 .

[51]  Anna Trosborg,et al.  Text typology and translation , 1997 .

[52]  N. Meshkati,et al.  Toward development of a cohesive model of workload , 1988 .

[53]  Hans P. Krings,et al.  Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes , 2001 .

[54]  Bernice E. Leary,et al.  What makes a book readable , 1935 .

[55]  R. Lyman Ott.,et al.  An introduction to statistical methods and data analysis , 1977 .

[56]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[57]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[58]  K. A. Ericsson,et al.  Protocol Analysis and Expert Thought: Concurrent Verbalizations of Thinking during Experts' Performance on Representative Tasks , 2006 .

[59]  Riitta Jääskeläinen Think-aloud protocol studies into translation: An annotated bibliography , 2002 .

[60]  S. Ian Robertson,et al.  Problem-solving , 2001, Human Thinking.

[61]  Hasan Ayaz,et al.  Implementation of fNIRS for Monitoring Levels of Expertise and Mental Workload , 2011, HCI.

[62]  L. E. Sullivan,et al.  The SAGE glossary of the social and behavioral sciences , 2009 .

[63]  Paul J. Feltovich,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[64]  Erik Castello Text Complexity and Reading Comprehension Tests , 2008 .

[65]  S. Campbell,et al.  Translation and interpreting assessment in the context of educational measurement , 2003 .

[66]  Albrecht Neubert,et al.  Competence in Language, in Languages, and in Translation , 2000 .

[67]  Östen Dahl,et al.  The growth and maintenance of linguistic complexity , 2004 .

[68]  Wolfram Wilss,et al.  The science of translation : problems and methods , 1982 .

[69]  Lilo Moessner,et al.  Genre, Text Type, Style, Register: A Terminological Maze? , 2001 .

[70]  Anthony Pym,et al.  Redefining Translation Competence in an Electronic Age. In Defence of a Minimalist Approach , 2004 .

[71]  K. Rayner,et al.  The psychology of reading , 1989 .

[72]  Ronald P. Carver,et al.  Measuring Prose Difficulty Using the Rauding Scale. , 1975 .

[73]  B. Hatim,et al.  The Translator As Communicator , 1997 .

[74]  Eva Lindgren,et al.  Computer keystroke logging and writing: methods and applications , 2006 .

[75]  Stuart Campbell,et al.  The interaction between text difficulty and translation accuracy , 2002 .

[76]  Waldemar Karwowski Workload and Electro-Encephalography Dynamics , 2006 .

[77]  R. Flesch A new readability yardstick. , 1948, The Journal of applied psychology.

[78]  Arthur C. Graesser,et al.  Methods of Automated Text Analysis , 2010 .

[79]  K. A. Ericsson,et al.  Protocol analysis: Verbal reports as data, Rev. ed. , 1993 .

[80]  Douglas Biber,et al.  Variation across speech and writing: Methodology , 1988 .