Visual signaling in virtual world-based assessments: The SAVE Science project

In this paper we describe a study into the impact of visual signaling techniques used in a virtual world-based assessment of science inquiry and content on (1) student cognitive load and (2) assessment efficiency. The study, run with 7th grade students in the United States, found that use of visual signaling was significantly associated with lower levels of student self-reported cognitive load versus students in a no-signaling version of the assessment. Further, the efficiency of the virtual world-based assessment was significantly higher, as measured by in-world object interaction rates, for students in the visual signaling version of the assessment than for those in the no-signaling treatment. In the paper, we discuss the results and their meaning for the design of virtual world and game-based assessments.

[1]  R. Moreno.  Optimising learning from animations by minimising cognitive load: cognitive and affective consequences of signalling and segmentation methods , 2007 .

[2]  Chris Lawrence.  Take a load off: cognitive considerations for game design , 2006 .

[3]  H. Schweingruber,et al.  America's lab report : investigations in high school science , 2006 .

[4]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[5]  F. Paas,et al.  Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach , 1994 .

[6]  Andrew Morozov.  The Effects of Spatial Visualization Ability and Graphical Navigational Aids on Cognitive Load and Learning from Web-Based Instruction , 2009 .

[7]  Richard E. Mayer,et al.  Signaling as a Cognitive Guide in Multimedia Learning , 2001 .

[8]  Brian C. Nelson,et al.  Presence and Middle School Students' Participation in a Virtual Game Environment to Assess Science Inquiry , 2012, J. Educ. Technol. Soc..

[9]  J. Sweller.  COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[10]  Richard E. Mayer,et al.  Unique Contributions of Eye-Tracking Research to the Study of Learning with Graphics , 2010 .

[11]  Brian C. Nelson,et al.  Exploring Cognitive Load in Immersive Educational Games: The SAVE Science Project , 2010, Int. J. Gaming Comput. Mediat. Simulations.

[12]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[13]  Brian C. Nelson,et al.  Global channels of evidence for learning and assessment in complex game environments , 2011, Br. J. Educ. Technol..

[14]  Brian C. Nelson,et al.  Managing cognitive load in educational multi-user virtual environments: reflection on design practice , 2008 .

[15]  F. Paas,et al.  How to Optimize Learning From Animated Models: A Review of Guidelines Based on Cognitive Load , 2008 .

[16]  Joel Michael,et al.  Conceptual assessment in the biological sciences: a National Science Foundation-sponsored workshop. , 2007, Advances in physiology education.

[17]  P. Chandler,et al.  The Role of Visual Indicators in Dual Sensory Mode Instruction , 1997 .

[18]  A. Paivio.  Mental Representations: A Dual Coding Approach , 1986 .

[19]  Brian C. Nelson,et al.  Using immersive virtual environments to assess science content understanding: The impact of context , 2010, CELDA 2010.

[20]  James C. Lester,et al.  The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents? , 2001 .

[21]  Hee-Sun Lee,et al.  Research towards an expanded understanding of inquiry science beyond one idealized standard , 2003 .

[22]  Brian C. Nelson,et al.  Designing for Real-World Inquiry in Virtual Environments , 2006 .

[23]  J. Sweller.  Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load , 2010 .

[24]  Chwen Jen Chen,et al.  Guiding Exploration Through Three-Dimensional Virtual Environments: A Cognitive Load Reduction Approach , 2008 .

[25]  H. Tabbers,et al.  Multimedia instructions and cognitive load theory: effects of modality and cueing. , 2004, The British journal of educational psychology.

[26]  Brian C. Nelson.  Exploring the Use of Individualized, Reflective Guidance In an Educational Multi-User Virtual Environment , 2007 .

[27]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[28]  Susan R. Goldman,et al.  Learning in complex domains: When and why do multiple representations help? , 2003 .

[29]  A. Paivio.  Dual coding theory: Retrospect and current status. , 1991 .

[30]  R. Mayer.  Cognitive Theory and the Design of Multimedia Instruction: An Example of the Two‐Way Street Between Cognition and Instruction , 2002 .

[31]  Günter Daniel Rey.  Reading direction and signaling in a simple computer simulation , 2010, Comput. Hum. Behav..

[32]  F. Paas,et al.  Attention Cueing as a Means to Enhance Learning from an Animation , 2007 .

[33]  Richard Lowe,et al.  Animation and learning: selective processing of information in dynamic graphics , 2003 .

[34]  F. Paas,et al.  Attention guidance in learning from a complex animation: Seeing is understanding? , 2010 .

[35]  Brian C. Nelson,et al.  Scientific Inquiry in Educational Multi-user Virtual Environments , 2007 .

[36]  S. Barab,et al.  Eat Your Vegetables and Do Your Homework: A Design­ Based Investigation of Enjoyment and Meaning in Learning , 2005 .

[37]  James Paul Gee,et al.  Looking where the light is bad: Video games and the future of assessment , 2010 .

[38]  Russell G. Almond,et al.  AN ASSESSMENT FOR LEARNING SYSTEM CALLED ACED: DESIGNING FOR LEARNING EFFECTIVENESS AND ACCESSIBILITY , 2007 .

[39]  Fred Paas,et al.  Interdisciplinary Perspectives Inspiring a New Generation of Cognitive Load Research , 2009 .

[40]  Brian C. Nelson,et al.  Virtual environments for situated science assessment , 2009, CELDA 2009.