Sound as Affective Design Feature in Multimedia Learning--Benefits and Drawbacks from a Cognitive Load Theory Perspective.

The study presented in this paper investigates the potential effects of including non-speech audio such as sound effects into multimedia-based instruction taking into account Sweller’s cognitive load theory (Sweller, 2005) and applied frameworks such as the cognitive theory of multimedia learning (Mayer, 2005) and the cognitive affective theory of learning with media (Moreno, 2006). Proceeding from the assumption that sound is an incisive means to affect people’s emotional state it is argued that sound may also be well suited to stimulate involvement and motivation in learning situations, thereby bringing the learner to invest more mental effort into learning, which finally leads to better learning performance. This paper refers to an experimental case study, which was carried out within the framework of a Master’s Thesis at the University of Applied Sciences Bremerhaven (Germany). In order to investigate the cognitive effects of including sound into multimedia learning, two groups of 1st semester Digital Media students were asked to learn about a historic subject using two different experimental designs: One version of a prototypical learning application consists of a photo slideshow with accompanying audio narration and another version consists of the same material supplemented with environmental sounds that illustrate the content of the lesson. Comparing both groups, the results don’t reveal significant differences in learning performance. However, the subjective mental effort ratings of the participants are identified as a positive predictor for the performance score and are thus hypothetically discussed as being an indicator for learner motivation. The analysis finally confirms that the learner involvement, which is a measure relating the performance score and the mental effort ratings (Paas et al., 2005), during the subsequent achievement test is significantly higher when sounds were presented during instruction. These results suggest that the inclusion of sound may have positive effects on motivation and learning, which is according to a cognitive-motivational theory.

[1]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[2]  Richard E. Mayer,et al.  Cognitive Theory of Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.

[3]  Michel Chion,et al.  Audio-Vision: Sound on Screen , 1994 .

[4]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[5]  Richard E. Mayer,et al.  The Cambridge Handbook of Multimedia Learning: Cognitive Theory of Multimedia Learning , 2005 .

[6]  Denis Smalley,et al.  The listening imagination: Listening in the electroacoustic era , 1996 .

[7]  Benjamin Naumann,et al.  Mental Representations A Dual Coding Approach , 2016 .

[8]  F. Paas,et al.  A motivational perspective on the relation between mental effort and performance: Optimizing learner involvement in instruction , 2005 .

[9]  J. Sweller.  Implications of Cognitive Load Theory for Multimedia Learning , 2005, The Cambridge Handbook of Multimedia Learning.

[10]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[11]  F. Thomas,et al.  Disney Animation: The Illusion of Life , 1981 .

[12]  S. McAdams,et al.  Auditory Cognition. (Book Reviews: Thinking in Sound. The Cognitive Psychology of Human Audition.) , 1993 .

[13]  R. Mayer,et al.  Interactive Multimodal Learning Environments Special Issue on Interactive Learning Environments: Contemporary Issues and Trends , 2007 .

[14]  R. Mayer,et al.  A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. , 2000 .

[15]  R. Mayer,et al.  Cognitive constraints on multimedia learning: When presenting more material results in less understanding. , 2001 .

[16]  Glen Holmes,et al.  The importance of culture when creating audio-enhanced, web-based instruction , 2002 .

[17]  Ward Mitchell Cates,et al.  Theoretical foundations for sound's use in multimedia instruction to enhance learning , 2001 .

[18]  J. Sweller.  The worked example effect and human cognition , 2006 .

[19]  Ward Mitchell Cates,et al.  Sound’s use in instructional software to enhance learning: a theory-to-practice content analysis , 2008 .

[20]  Roland Brünken,et al.  Does cognitive load moderate the seductive details effect? A multimedia study , 2011, Comput. Hum. Behav..

[21]  R. Moreno,et al.  Does the modality principle hold for different media? A test of the method-affects-learning hypothesis , 2006, J. Comput. Assist. Learn..

[22]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[23]  A. Paivio,et al.  Memory for pictures and sounds: independence of auditory and visual codes. , 1994, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[24]  Eunjoon Rachel Um,et al.  Emotional design in multimedia learning. , 2012 .

[25]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[26]  Slava Kalyuga,et al.  Incorporating Learner Experience into the Design of Multimedia Instruction. , 2000 .

[27]  Sharon K Tindall-Ford,et al.  Learning by imagining. , 2001, Journal of experimental psychology. Applied.