Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the modality effect

The purpose of this study was to investigate the effects of modality (written text vs. spoken text) and visual cueing (low cueing vs. high cueing) on the learning and mental effort of participants studying a computer-based static diagram at their own pace. Participants were randomly assigned to four versions of the computer-based materials formed into a 2x2 factorial design by crossing modality with cueing. The results revealed a reverse modality effect, wherein participants studying written text outperformed those studying spoken text on tests of free recall, matching, comprehension, and spatial recall, but not mental effort. Information cueing did not significantly affect either performance or mental effort. These findings are discussed in the context of two popular explanations of the modality effect: the cognitive resources explanation and the perceptual resources explanation. The results were best explained from a perceptual resources viewpoint.

[1]  Robert K. Atkinson,et al.  Using animations and visual cueing to support learning of scientific concepts and processes , 2011, Comput. Educ..

[2]  A. Colman,et al.  Modality of Communication and Recall of Health-related Information , 1997, Journal of health psychology.

[3]  A. Kooijman,et al.  Using route and survey information to generate cognitive maps: differences between normally sighted and visually impaired individuals , 2009 .

[4]  Roland Brünken,et al.  Working Memory Interference During Processing Texts and Pictures: Implications for the Explanation of the Modality Effect , 2010 .

[5]  Barrie Gunter,et al.  Memory for Material Presented in the Media: The Superiority of Written Communication , 1988 .

[6]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[7]  M. Hegarty.  Mental animation: inferring motion from static displays of mechanical systems. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Wolfgang Schnotz,et al.  Structural Inferences in Reading and Listening , 1985 .

[9]  J. Sweller,et al.  Reducing cognitive load by mixing auditory and visual presentation modes , 1995 .

[10]  R. Mayer,et al.  Cognitive Principles of Multimedia Learning: The Role of Modality and Contiguity , 1999 .

[11]  Florian Schmidt-Weigand,et al.  A closer look at split visual attention in system- and self-paced instruction in multimedia learning , 2010 .

[12]  Robert H. Logie,et al.  Visual Similarity Effects in Immediate Verbal Serial Recall , 2000, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[13]  Jochen Müsseler,et al.  Recalling oral and written discourse. , 1987 .

[14]  Slava Kalyuga,et al.  Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning , 2005 .

[15]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[16]  Dennis Norris,et al.  Visual similarity effects on masked priming , 2014, Memory & cognition.

[17]  John Sweller,et al.  The Split-Attention Principle in Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.

[18]  Barrie Gunter,et al.  The primacy of print: immediate cued recall of news as a function of the channel of communication , 1989 .

[19]  John Sweller,et al.  The Modality Effect , 2011 .

[20]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[21]  Larry L. Jacoby,et al.  Cross-modal feature and conjunction errors in recognition memory , 2001 .

[22]  M. D’Esposito.  Working memory. , 2008, Handbook of clinical neurology.

[23]  Stuart Norcross.  The Derivation of a New , 2003 .

[24]  F. Paas,et al.  Attention guidance in learning from a complex animation: Seeing is understanding? , 2010 .

[25]  C. Bazerman.  Physicists Reading Physics , 1985 .

[26]  Paul Ginns.  Meta-Analysis of the Modality Effect. , 2005 .

[27]  W. Schnotz.  Integrated Model of Text and Picture Comprehension , 2021, The Cambridge Handbook of Multimedia Learning.

[28]  Fred Paas,et al.  Exploring Multidimensional Approaches to the Efficiency of Instructional Conditions , 2004 .

[29]  J. Sweller,et al.  Cognitive load theory, modality of presentation and the transient information effect. , 2011 .

[30]  Patricia D. Mautone,et al.  Cognitive Aids for Guiding Graph Comprehension. , 2007 .

[31]  Wolfgang Schnotz,et al.  The Cambridge Handbook of Multimedia Learning: An Integrated Model of Text and Picture Comprehension , 2005 .

[32]  Katharina Scheiter,et al.  The modality effect in multimedia learning: theoretical and empirical limitations , 2008, ICLS.

[33]  Anne Schueler,et al.  Does a lack of contiguity with visual text cause the modality effect in multimedia learning , 2008 .

[34]  Steven M. Smith,et al.  Effects of perceptual modality on verbatim and gist memory , 2004, Psychonomic bulletin & review.

[35]  R. Curtis,et al.  Designing health communication: Testing the explanations for the impact of communication medium on effectiveness , 2000 .

[36]  F. Paas,et al.  Towards a Framework for Attention Cueing in Instructional Animations: Guidelines for Research and Design , 2009 .

[37]  H. Tabbers,et al.  Multimedia instructions and cognitive load theory: effects of modality and cueing. , 2004, The British journal of educational psychology.

[38]  Katharina Scheiter,et al.  The perceptual basis of the modality effect in multimedia learning. , 2011, Journal of experimental psychology. Applied.

[39]  Paul Junor,et al.  A comparison of reading and listening performance: computer-controlled speech and printed text presentations , 1994, Br. J. Educ. Technol..

[40]  A. Furnham,et al.  Remembering science: The recall of factual information as a function of the presentation mode , 1990 .

[41]  Adrian Furnham.  Remembering Stories as a Function of the Medium of Presentation , 2001, Psychological reports.

[42]  Barrie Gunter,et al.  Children's and adults' recall of children's news stories in both print and audio-visual presentation modalities , 2002 .

[43]  Mary Hegarty,et al.  Top-down and bottom-up influences on learning from animations , 2007, Int. J. Hum. Comput. Stud..

[44]  F. Paas.  Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. , 1992 .

[45]  John Sweller,et al.  The Cambridge Handbook of Multimedia Learning: The Split-Attention Principle in Multimedia Learning , 2005 .