The modality effect in multimedia instructions

The modality effect in multimedia instructions Huib K. Tabbers (huib.tabbers@ou.nl) Rob L. Martens (rob.martens@ou.nl) Jeroen J. G. van Merrienboer (jeroen.vanmerrienboer@ou.nl) Open University of the Netherlands: Educational Technology Expertise Centre; P.O. Box 2960 NL-6401 DL Heerlen, The Netherlands Abstract The influence of presentation format on the effectiveness of multimedia instructions was investigated. According to Cognitive Load Theory (Sweller, Van Merrienboer & Paas, 1998) and Mayer's theory of multimedia learning (Moreno & Mayer, 1999), replacing visual text with audio will decrease working memory load and improve learning (modality effect). This hypothesis was tested in two experiments in which students studied multimedia instructions on an instructional design model. The students reported the mental effort spent on the instructions, and made a retention and a transfer test after the instructions. The results show that replacing text with audio is only effective when multimedia instructions are system-paced. Introduction Guidelines for the design of multimedia instructions are often based on intuition and practical experience rather than on the results of experimental research (Park & Hannafin, 1994). However, two recent lines of research that have yielded some interesting results are the work by John Sweller and his colleagues on Cognitive Load Theory (Sweller, 1988; Sweller, van Merrienboer & Paas, 1998), and the experiments carried out by Richard Mayer and his colleagues on multimedia learning (for an overview, see Moreno & Mayer, 1999). Both researchers claim that multimedia instructions consisting of verbal and pictorial information, like for example a picture of a machine and a text about its functioning, place a high demand on working memory resources, because the learner has to switch between text and picture in order to integrate them mentally. An interesting finding in their research is that this memory load can be reduced by presenting the verbal information auditorily instead of visually. They call this phenomenon the modality effect or modality principle. The explanation they give is based on the working memory model of Baddeley (1992). In his model, working memory has two modality-specific slave systems: one for processing visual and spatial information and one for acoustic information. When information is presented in two sensory modalities (visual and auditory) rather than one, both slave systems are addressed and total working memory capacity is increased. So relative to the available resources, the memory load of the multimedia instructions is reduced, leaving more space for the actual learning process. Sweller and Mayer have demonstrated the superiority of audio over written or on-screen text in a number of experiments. For example, Jeung, Chandler and Sweller (1997) and Mousavi, Low and Sweller (1995) showed that students receiving multimedia instructions with audio spent less time on subsequent problem solving compared to students receiving visual-only instructions. Furthermore, students in experiments by Kalyuga, Chandler and Sweller (1999) and Tindall-Ford, Chandler and Sweller (1997) reported less mental effort during instruction and attained higher test scores, while in the studies by Mayer and Moreno (1998; 1999) students had higher scores on retention, transfer and matching tests. In one experiment, Moreno and Mayer (1999) even used instructions in which the animation and the accompanying text were presented sequentially instead of simultaneously. Despite the temporal detachment of text and picture, bimodal instructions still proved to be superior to visual-only instructions. This shows that the modality effect seems to be at least for some part the result of an increase in available memory resources. Based on these results, Sweller and Mayer strongly advocate the use of audio in multimedia instructions. However, one limitation of the above-mentioned studies is that they all deal with short multimedia instructions on well-defined technical subjects like geometry (Mousavi et al., 1995; Jeung et al., 1997), scientific explanations of how lightning develops (Mayer & Moreno, 1998; Moreno & Mayer, 1999) and electrical engineering (Kalyuga et al., 1999; Tindall-Ford et al., 1997). This raises the question how powerful the modality effect actually is. Can it also be demonstrated with multimedia instructions that are outside the technical domain and are of greater length? This question is dealt with in the first experiment of this study. The second issue that can be raised given the evidence so far, is that the results can be explained in more than one way. For example, Jeung et al. (1997), Mousavi et al. (1995) and Tindall-Ford et al. (1997) used visual-only instructions in which the complete explanatory text was printed next to the diagram and