How Should We Evaluate Multimedia Learning Environments

Early research with multimedia environments questioned whether these environments are effective in supporting learning. More recently it has been acknowledged that this question should really be about the specific conditions and reasons why multimedia is effective. However, while the argument has become more sophisticated, the techniques for evaluating learning with multimedia environments have not always followed suit. The dominant approach at present involves factorial designs with novices as participants, learning something for a short period of time with outcomes tested by an immediate pen and paper post-test. In this chapter, the positive aspects of this approach are reviewed, but it will be argued that such an approach limits the questions that can be answered. Four important such questions about learning with multimedia are proposed and then the chapter describes a range of methodologies that can be used to answer them.

[1]  P. Chandler,et al.  THE SPLIT‐ATTENTION EFFECT AS A FACTOR IN THE DESIGN OF INSTRUCTION , 1992 .

[2]  Perry D. Klein,et al.  Rethinking the multiplicity of cognitive resources and curricular representations: Alternatives to 'learning styles' and 'multiple intelligences' , 2003 .

[3]  Shaaron Ainsworth,et al.  The effects of self‐explaining when learning with text or diagrams , 2003 .

[4]  Michal Yerushalmy,et al.  Student perceptions of aspects of algebraic function using multiple representation software , 1991 .

[5]  Tina Seufert.  Supporting Coherence Formation in Learning from Multiple Representations , 2003 .

[6]  Hermann G. Ebner,et al.  Improving cross-content transfer in text processing by means of active graphical representation , 2003 .

[7]  C. Hood,et al.  Education and Intelligence. , 1912 .

[8]  Richard E. Mayer,et al.  Multimedia Learning: The Promise of Multimedia Learning , 2001 .

[9]  R. Mayer,et al.  For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. , 1994 .

[10]  Jan van der Meij,et al.  Learning with Multiple Representations , 1998 .

[11]  Shaaron Ainsworth,et al.  Examining the Effects of Different Multiple Representational Systems in Learning Primary Mathematics , 2002 .

[12]  S. Ainsworth.  DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[13]  Padraic Monaghan,et al.  Proceedings of the 23rd annual conference of the cognitive science society , 2001 .

[14]  C. Howe,et al.  Conceptual knowledge in physics: The role of group interaction in computer-supported teaching , 1992 .

[15]  Slava Kalyuga,et al.  Managing split-attention and redundancy in multimedia instruction , 1999 .

[16]  Hermina J. M. Tabachneck-Schijf,et al.  CaMeRa: A Computational Model of Multiple Representations , 1997, Cogn. Sci..

[17]  R. Siegler,et al.  How Does Change Occur: A Microgenetic Study of Number Conservation , 1995, Cognitive Psychology.

[18]  Richard Lowe,et al.  Animation and learning: selective processing of information in dynamic graphics , 2003 .

[19]  Fernand Gobet,et al.  CHREST+: A simulation of how humans learn to solve problems using diagrams , 2000 .

[20]  R. Kozma,et al.  The Roles of Representations and Tools in the Chemistry Laboratory and Their Implications for Chemistry Learning , 2000 .

[21]  R. Mayer,et al.  Multimedia Learning: The Promise of Multimedia Learning , 2001 .

[22]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[23]  Pablo Romero,et al.  The effects of graphical and textual visualisations in multi-representational debugging environments , 2003, IEEE Symposium on Human Centric Computing Languages and Environments, 2003. Proceedings. 2003.

[24]  Peter Reimann,et al.  Multimedia learning: beyond modality , 2003 .

[25]  R. Glaser.  Advances in Instructional Psychology , 1978 .

[26]  C. Tsui,et al.  Genetics Reasoning with Multiple External Representations , 2003 .

[27]  Albert T. Corbett,et al.  Assessing dynamics in computer-based instruction , 1996, CHI.

[28]  Bill Winn,et al.  Charts, Graphs, and Diagrams in Educational Materials , 1987 .

[29]  D. Lewalter.  Cognitive Strategies for Learning from Static and Dynamic Visuals , 2003 .

[30]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[31]  Jeroen J. G. van Merriënboer,et al.  The modality effect in multimedia instructions , 2001 .

[32]  Marsha C. Lovett,et al.  Modeling individual differences in working memory performance: a source activation account , 2001, Cogn. Sci..

[33]  Lawrence J. Najjar.  Principles of Educational Multimedia User Interface Design , 1998, Hum. Factors.

[34]  Wolff-Michael Roth,et al.  Professionals Read Graphs: A Semiotic Analysis , 2001 .

[35]  Wolfgang Schnotz.  Sign systems, technologies, and the acquisition of knowledge , 2001 .

[36]  Susan R. Goldman,et al.  Learning in complex domains: When and why do multiple representations help? , 2003 .

[37]  Lloyd P. Rieber,et al.  Animation in computer-based instruction , 1990 .

[38]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[39]  A. diSessa.  Metarepresentation: Native Competence and Targets for Instruction , 2004 .

[40]  Barbara C. Buckley.  Interactive multimedia and model-based learning in biology , 2000 .

[41]  Joan K. Gallini,et al.  When Is an Illustration Worth Ten Thousand Words , 1990 .

[42]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[43]  P. Cheng.  Electrifying diagrams for learning: principles for effective representational systems , 2002 .

[44]  Jean-Michel Passerault,et al.  Analyzing learner-hypermedia interaction: An overview of online methods , 1999 .

[45]  D. J. Gilmore,et al.  Individual differences and strategy selection in reasoning , 1997 .

[46]  Jean-François Rouet,et al.  Multimedia learning : cognitive and instructional issues , 2001 .

[47]  Sara Price.  Diagram Representation: The Cognitive Basis for Understanding Animation in Education , 2002 .

[48]  R. Mayer,et al.  A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. , 2000 .

[49]  Stephan Schwan,et al.  Learning with Hyperlinked Videos--Design Criteria and Efficient Strategies for Using Audiovisual Hypermedia. , 2004 .

[50]  Peter C.-H. Cheng,et al.  Electrifying diagrams for learning: principles for complex representational systems , 2002, Cogn. Sci..