Coordinating Visual and Auditory Cueing in Multimedia Learning

Providing single-modality cueing (either visual cueing or auditory cueing) in multimedia lessons does not consistently improve learning outcomes. In 3 eye-tracking experiments, some students learned an onscreen lesson with an oral explanation of graphics and then took a posttest on the material (no cues group). Across all 3 experiments, students spent more time attending to the relevant portion of the graphic and performed better on posttests if coordinated cues were added to the lesson (coordinated dual cues group), in which key elements were spoken with deeper intonation (auditory cue) at the same time the element turned red in the graphic (visual cue). Presenting coordinated visual and auditory cues also resulted in better posttest performance than presenting only a visual cue (visual-only cues group) or auditory cue alone (auditory-only cues group) in Experiment 1, or presenting visual and auditory cues that were unmatched (mismatched dual cues group) in Experiment 2 or unsynchronized (visual-before-auditory cues group and visual-after-auditory cues group) in Experiment 3. These findings extend and sharpen the signaling principle concerning how best to highlight instructional material in multimedia learning, using coordinated auditory and visual cues.

[1]  Richard E. Mayer,et al.  Cognitive Theory of Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.

[2]  Richard E. Mayer,et al.  Multimedia Learning: PRINCIPLES FOR FOSTERING GENERATIVE PROCESSING IN MULTIMEDIA LEARNING , 2001 .

[3]  R. Mayer.  How Can Brain Research Inform Academic Learning and Instruction? , 2017 .

[4]  Michael S. C. Thomas,et al.  The principles and practices of educational neuroscience: Comment on Bowers (2016). , 2016, Psychological review.

[5]  A. Linn.  4.6 ‘Top down’ and ‘bottom up’ influences and behaviours , 2016 .

[6]  Xie Heping,et al.  Cueing effect in multimedia learning: A meta-analysis , 2016 .

[7]  Roger Bruning,et al.  Exploring effects of background context familiarity and signaling on comprehension, recall, and cognitive load , 2016 .

[8]  J. Bowers.  The practical and principled problems with educational neuroscience. , 2016, Psychological review.

[9]  Eric Jamet,et al.  Enhancing interactive tutorial effectiveness through visual cueing , 2016 .

[10]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[11]  Katharina Scheiter,et al.  Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis , 2016 .

[12]  Zhang Yi,et al.  The Cognitive and Neural Mechanism of Text and Picture Processing in Multimedia Learning , 2015 .

[13]  K. Scheiter,et al.  Signals foster multimedia learning by supporting integration of highlighted text and diagram elements , 2015 .

[14]  Peter Gerjets,et al.  Watching corresponding gestures facilitates learning with animations by activating human mirror-neurons: An fNIRS study , 2015 .

[15]  Zhou Zong-kui,et al.  The Spatial Contiguity Effect in Multimedia Learning: The Role of Cueing , 2015 .

[16]  Jennifer L. Chiu,et al.  Evidence for effective uses of dynamic visualisations in science curriculum materials , 2015 .

[17]  Yi-Huang Su.  Content congruency and its interplay with temporal synchrony modulate integration between rhythmic audiovisual streams , 2014, Front. Integr. Neurosci..

[18]  Yu-Cin Jian,et al.  Learners' eye movements during construction of mechanical kinematic representations from static diagrams , 2014 .

[19]  T. Gog.  The signaling (or cueing) principle in multimedia learning , 2014 .

[20]  Eric Jamet,et al.  An eye-tracking study of cueing effects in multimedia learning , 2014, Comput. Hum. Behav..

[21]  Scotty D. Craig,et al.  Virtual Humans and Gesturing during Multimedia Learning: An Investigation of Predictions from the Temporal Contiguity Effect , 2013 .

[22]  Katharina Scheiter,et al.  Learning about locomotion patterns: Effective use of multiple pictures and motion-indicating arrows , 2013, Comput. Educ..

[23]  T. Gog,et al.  Learning to see: Guiding students' attention via a Model's eye movements fosters learning , 2013 .

[24]  Jean-Michel Boucheix,et al.  Cueing animations: Dynamic signaling aids information extraction and comprehension , 2013 .

[25]  Ismahan Arslan,et al.  Examining the effects of cueing and prior knowledge on learning, mental effort, and study time in a complex animation , 2013 .

[26]  C. Beauchamp,et al.  Boundary as Bridge: An Analysis of the Educational Neuroscience Literature from a Boundary Perspective , 2013 .

[27]  A. Renkl,et al.  Learning from Narrated Animations with Different Support Procedures: Working Memory Capacity Matters , 2012 .

[28]  Steven M. Crooks,et al.  Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the modality effect , 2012, Comput. Hum. Behav..

[29]  Salam Sazilah,et al.  The Effects of Visual Cues and Learners’ Field Dependence in Multiple External Representations Environment for Novice Program Comprehension , 2012 .

[30]  Slava Kalyuga,et al.  Educational Implications of Expertise Reversal Effects in Learning and Performance of Complex Cognitive and Sensorimotor Skills , 2012 .

[31]  Richard Lowe,et al.  Cueing complex animations: Does direction of attention foster learning processes? , 2011 .

[32]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[33]  C. Spence.  Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[34]  Fred Paas,et al.  Attention cueing in an instructional animation: The role of presentation speed , 2011, Comput. Hum. Behav..

[35]  Thomas Huk,et al.  The educational value of visual cues and 3D-representational format in a computer animation under restricted and realistic conditions , 2010 .

[36]  F. Paas,et al.  Attention guidance in learning from a complex animation: Seeing is understanding? , 2010 .

[37]  Richard K. Lowe,et al.  An Eye Tracking Comparison of External Pointing Cues and Internal Continuous Cues in Learning with Complex Animations , 2010 .

[38]  David B. Pisoni,et al.  Neural processing of asynchronous audiovisual speech perception , 2010, NeuroImage.

[39]  Erol Özçelik,et al.  Why does signaling enhance multimedia learning? Evidence from eye movements , 2010, Comput. Hum. Behav..

[40]  Erol Özçelik,et al.  An eye-tracking study of how color coding affects multimedia learning , 2009, Comput. Educ..

[41]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[42]  F. Paas,et al.  Towards a Framework for Attention Cueing in Instructional Animations: Guidelines for Research and Design , 2009 .

[43]  A. Renkl,et al.  Instructional Aids to Support a Conceptual Understanding of Multiple Representations. , 2009 .

[44]  M. D’Esposito.  Working memory. , 2008, Handbook of clinical neurology.

[45]  D. Leutner,et al.  Instructional animation versus static pictures: A meta-analysis , 2007 .

[46]  Richard Lowe,et al.  Learning with Animation: Research Implications for Design , 2007 .

[47]  Mary Hegarty,et al.  Top-down and bottom-up influences on learning from animations , 2007, Int. J. Hum. Comput. Stud..

[48]  Slava Kalyuga.  Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction , 2007 .

[49]  F. Paas,et al.  Attention Cueing as a Means to Enhance Learning from an Animation , 2007 .

[50]  Kai Sassenberg,et al.  Why some groups just feel better: the regulatory fit of group power. , 2007, Journal of personality and social psychology.

[51]  Rainer Goebel,et al.  The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. , 2006, Cerebral cortex.

[52]  Logan Fiorella,et al.  Principles for Reducing Extraneous Processing in Multimedia Learning: Coherence, Signaling, Redundancy, Spatial Contiguity and Temporal Contiguity Principles. , 2014 .

[53]  Mireille Betrancourt,et al.  The Cambridge Handbook of Multimedia Learning: The Animation and Interactivity Principles in Multimedia Learning , 2005 .

[54]  Marty G. Woldorff,et al.  Selective Attention and Multisensory Integration: Multiple Phases of Effects on the Evoked Brain Activity , 2005, Journal of Cognitive Neuroscience.

[55]  Paul J. Laurienti,et al.  Semantic congruence is a critical factor in multisensory behavioral performance , 2004, Experimental Brain Research.

[56]  H. Tabbers,et al.  Multimedia instructions and cognitive load theory: effects of modality and cueing. , 2004, The British journal of educational psychology.

[57]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[58]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[59]  Richard E. Mayer,et al.  Signaling as a Cognitive Guide in Multimedia Learning , 2001 .

[60]  Belita Gordon,et al.  Using Rating Augmentation to Expand the Scale of an Analytic Rubric , 2000 .

[61]  Slava Kalyuga,et al.  Managing split-attention and redundancy in multimedia instruction , 1999 .

[62]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[63]  P. Chandler,et al.  The Role of Visual Indicators in Dual Sensory Mode Instruction , 1997 .

[64]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[65]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[66]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.