Measuring Cognitive Load in the Presence of Educational Video: Towards a Multimodal Methodology.

The use of video has become well established in education, from traditional courses to blended and online courses. It has grown both in its diversity of applications as well as its content. Such educational video however is not fully accessible to all students, particularly those who require additional visual support or students studying in a foreign language. Subtitles (also known as captions) represent a unique solution to these language and accessibility barriers, however, the impact of subtitles on cognitive load in such a rich and complex multimodal environment has yet to be determined. Cognitive load is a complex construct and its measurement by means of single indirect and unidimensional methods is a severe methodological limitation. Building upon previous work from several disciplines, this paper moves to establish a multimodal methodology for the measurement of cognitive load in the presence of educational video. We show how this methodology, with refinement, can allow us to determine the effectiveness of subtitles as a learning support in educational contexts. This methodology will also make it possible to analyse the impact of other multimedia learning technology on cognitive load.

[1]  John Sweller,et al.  Redundancy in foreign language reading comprehension instruction: Concurrent written and spoken presentations , 2007 .

[2]  Philip J. Guo,et al.  How video production affects student engagement: an empirical study of MOOC videos , 2014, L@S.

[3]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[4]  Stephen Doherty,et al.  Eye tracking as an MT evaluation technique , 2010, Machine Translation.

[5]  F. Hutzler,et al.  Systematic influence of gaze position on pupil size measurement: analysis and correction , 2011, Behavior research methods.

[6]  Anna Curtis Chandler,et al.  Audio-Visual Techniques for Enrichment of the Curriculum , 2015 .

[7]  J. Sweller,et al.  Reducing cognitive load by mixing auditory and visual presentation modes , 1995 .

[8]  Glenn F. Wilson,et al.  Psychophysiological responses to changes in workload during simulated air traffic control , 1996, Biological Psychology.

[9]  John N. Williams,et al.  The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling , 2002 .

[10]  Slava Kalyuga,et al.  Instructional benefits of spoken words: A review of cognitive load factors , 2012 .

[11]  Paul Markham,et al.  Captioned Videotapes and Second‐Language Listening Word Recognition , 1999 .

[12]  Stephen Doherty,et al.  Towards a cognitive audiovisual translatology: subtitles and embodied cognition , 2016 .

[13]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[14]  Michela Ott,et al.  The potential relevance of cognitive neuroscience for the development and use of technology-enhanced learning , 2015 .

[15]  Jan-Louis Kruger,et al.  Subtitles and Eye Tracking: Reading and Performance , 2014 .

[16]  R. Mayer,et al.  Verbal redundancy in multimedia learning: When reading helps listening , 2002 .

[17]  R Goldstein,et al.  Effect of task difficulty and interstimulus interval on blink parameters. , 1992, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[18]  J. Engelkamp.  Memory for actions , 1998 .

[19]  Patrick J Curran,et al.  Twelve Frequently Asked Questions About Growth Curve Modeling , 2010, Journal of cognition and development : official journal of the Cognitive Development Society.

[20]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[21]  T. Gog,et al.  Development of an instrument for measuring different types of cognitive load , 2013, Behavior Research Methods.

[22]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[23]  R. Mayer,et al.  Cognitive constraints on multimedia learning: When presenting more material results in less understanding. , 2001 .

[24]  R. Mayer,et al.  Maximizing Constructivist Learning From Multimedia Communications by Minimizing Cognitive Load , 1999 .

[25]  Stephen Doherty,et al.  Translations| The Impact of Translation Technologies on the Process and Product of Translation , 2016 .

[26]  F. Paas,et al.  Measurement of Cognitive Load in Instructional Research , 1994, Perceptual and motor skills.

[27]  L. Manelis,et al.  Effect of Awareness on an Indicator of Cognitive Load , 1979 .

[28]  D. Leutner,et al.  Assessment of cognitive load in multimedia learning using dual-task methodology. , 2002, Experimental psychology.

[29]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[30]  Sharon O'Brien,et al.  Assessing the Usability of Raw Machine Translated Output: A User-Centered Study Using Eye Tracking , 2014, Int. J. Hum. Comput. Interact..

[31]  A. Baddeley.  Human Memory: Theory and Practice, Revised Edition , 1990 .

[32]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[33]  Sylvain Sirois,et al.  Pupil diameter measurement errors as a function of gaze direction in corneal reflection eyetrackers , 2013, Behavior research methods.

[34]  Zenzi M. Griffin,et al.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[35]  Richard F. Schmid,et al.  The effects of technology use in postsecondary education: A meta-analysis of classroom applications , 2014, Comput. Educ..

[36]  Pavlo D. Antonenko,et al.  The influence of leads on cognitive load and learning in a hypertext environment , 2010, Comput. Hum. Behav..

[37]  J. Sweller.  Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load , 2010 .

[38]  R. Mayer,et al.  A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. , 2000 .

[39]  Pavlo D. Antonenko,et al.  Using Electroencephalography to Measure Cognitive Load , 2010 .

[40]  D. E. Irwin.  Fixation location and fixation duration as indices of cognitive processing , 2004 .

[41]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[42]  Slava Kalyuga.  Cognitive Load Theory: How Many Types of Load Does It Really Need? , 2011 .

[43]  Wendy Fox,et al.  Multimodal measurement of cognitive load during subtitle processing: same-language subtitles for foreign-language viewers , 2018 .

[44]  T. Gog,et al.  Effects of pairs of problems and examples on task performance and different types of cognitive load , 2014 .

[45]  Robin H. Kay,et al.  Exploring the use of video podcasts in education: A comprehensive review of the literature , 2012, Comput. Hum. Behav..

[46]  Morton Ann Gernsbacher,et al.  Video Captions Benefit Everyone , 2015, Policy insights from the behavioral and brain sciences.

[47]  K. Holyoak.  Mental representations. , 1982, Science.

[48]  Pat Hanrahan,et al.  Measuring the task-evoked pupillary response with a remote eye tracker , 2008, ETRA.

[49]  R. Vanderplank.  ‘Effects of’ and ‘effects with’ captions: How exactly does watching a TV programme with same-language subtitles make a difference to language learners? , 2013, Language Teaching.

[50]  Siyuan Chen,et al.  Eye activity as a measure of human mental effort in HCI , 2011, IUI '11.

[51]  I. Gere,et al.  Multimedia: Differences in cognitive processes observed with EEG , 1999 .