Interactive Science Multimedia and Visuospatial Processing

Diverse interactive multimedia products are developed for university students in health and natural sciences. These different multimedia tools allow exchanges of information with the students. In these exchanges, the students provide cognitive engagement, and the multimedia responds with feedback. The multimedia products that allow the highest levels of these interactions are simulations and videogames. We base this chapter on both multimedia tools and describe examples of effective high-level interactions. Nevertheless, we also note that too much interactivity can be detrimental, particularly for students with low visuospatial processing capacity. As predicted by cognitive load theory, these students may overwhelm their working memory capacity by trying to process the learning contents plus excessive interactive features. In short, properly designed simulations and videogames should seek a balance of productive engagement and responsive feedback. Examples of balanced multimedia have shown positive learning results across science topics. There is also accumulating evidence of the positive relationship between visuospatial processing and learning from interactive multimedia. As such, higher visuospatial processing abilities can foster learning from science simulations and understanding complex scientific visualizations. Also, playing videogames, particularly action videogames, can be effective for training visuospatial skills. We end this chapter by offering instructional implications and directions for future research.

[1]  Christian Gütl,et al.  Virtual laboratories for education in science, technology, and engineering: A review , 2016, Comput. Educ..

[2]  D. R. Montello,et al.  Learning a Spatial Skill for Surgery : How the Contributions of Abilities Change With Practice , 2006 .

[3]  John Sweller,et al.  Instructional Visualizations, Cognitive Load Theory, and Visuospatial Processing , 2019, Visuospatial Processing for Education in Health and Natural Sciences.

[4]  J. Castro-Alonso,et al.  Different Abilities Controlled by Visuospatial Processing , 2019, Visuospatial Processing for Education in Health and Natural Sciences.

[5]  R. Mayer,et al.  Learning Science in Immersive Virtual Reality , 2018, Journal of Educational Psychology.

[6]  R. Mayer,et al.  Learning as a Generative Activity: Eight Learning Strategies that Promote Understanding , 2015 .

[7]  Andrew T. Stull,et al.  Three Experimental Comparisons of Learner-generated versus Author-provided Graphic Organizers , 2007 .

[8]  Elizabeth O. Hayward,et al.  Emotional design in multimedia learning: Effects of shape and color on affect and learning , 2014 .

[9]  R. Mayer,et al.  Eight Ways to Promote Generative Learning , 2016 .

[10]  Richard E. Mayer,et al.  Multimedia Learning: INTRODUCTION TO MULTIMEDIA LEARNING , 2009 .

[11]  John Sweller,et al.  Undesirable Difficulty Effects in the Learning of High-Element Interactivity Materials , 2018, Front. Psychol..

[12]  James P. Birk,et al.  The role of visualization in learning from computer‐based images , 2005 .

[13]  P. Frensch,et al.  Effects of video game playing on measures of spatial performance: Gender effects in late adolescence. , 1994 .

[14]  Alexander P. Wind,et al.  Multimedia learning with computer games , 2014 .

[15]  J. Sweller,et al.  The Cambridge Handbook of Multimedia Learning: The Modality Principle in Multimedia Learning , 2005 .

[16]  R. Mayer,et al.  Paper-Based Aids for Learning with a Computer-Based Game , 2012 .

[17]  R. Mayer,et al.  Domain specificity of spatial expertise: The case of video game players. , 2002 .

[18]  Fred Paas,et al.  Investigating gender and spatial measurements in instructional animation research , 2018, Comput. Hum. Behav..

[19]  Herre van Oostendorp,et al.  Instructional Techniques to Facilitate Learning and Motivation of Serious Games , 2017 .

[20]  C. Kinzer,et al.  Foundations of Game-Based Learning , 2015 .

[21]  Evan M. Palmer,et al.  Intrinsic motivation and attentional capture from gamelike features in a visual search task , 2013, Behavior Research Methods.

[22]  Jan L. Plass,et al.  The cognitive impact of interactive design features for learning complex materials in medical education , 2014, Comput. Educ..

[23]  R. Mayer Computer Games in Education. , 2019, Annual review of psychology.

[24]  D. Uttal,et al.  Spatial Ability for University Biology Education , 2018, Advances in Intelligent Systems and Computing.

[25]  M. Chi,et al.  The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes , 2014 .

[26]  Fred Paas,et al.  Gender Imbalance in Instructional Dynamic Versus Static Visualizations: a Meta-analysis , 2019, Educational Psychology Review.

[27]  Douglas B. Clark,et al.  Digital Games, Design, and Learning , 2016, Review of educational research.

[28]  David Moreau,et al.  Differentiating two- from three-dimensional mental rotation training effects , 2013, Quarterly journal of experimental psychology.

[29]  Meng-Tzu Cheng,et al.  An educational game for learning human immunology: What do students learn and how do they perceive? , 2014, Br. J. Educ. Technol..

[30]  Jan L. Plass,et al.  Interactivity in multimedia learning: An integrated model , 2010, Comput. Hum. Behav..

[31]  John Sweller,et al.  Cognitive Load Theory for the Design of Medical Simulations , 2015, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[32]  R. Mayer,et al.  Game over for Tetris as a platform for cognitive skill training , 2018, Contemporary Educational Psychology.

[33]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[34]  Irene W. Y. Ma,et al.  Emotion, cognitive load and learning outcomes during simulation training , 2012, Medical education.

[35]  Kara J. Blacker,et al.  Effects of action video game training on visual working memory. , 2014, Journal of experimental psychology. Human perception and performance.

[36]  Maciek Skorko,et al.  Cognitive enhancement in video game players: The role of video game genre , 2015, Comput. Hum. Behav..

[37]  Jennifer J. Vogel-Walcutt,et al.  Applying the modality principle to real-time feedback and the acquisition of higher-order cognitive skills , 2012 .

[38]  Michele Jacobsen,et al.  Learning and assessment with images: A view of cognitive load through the lens of cerebral blood flow , 2017, Br. J. Educ. Technol..

[39]  Michael Barnett,et al.  Using Video Games to Support Pre-Service Elementary Teachers Learning of Basic Physics Principles , 2011 .

[40]  Andree Hartanto,et al.  Context counts: The different implications of weekday and weekend video gaming for academic performance in mathematics, reading, and science , 2017, Comput. Educ..

[41]  F. Gobet,et al.  Video Game Training Does Not Enhance Cognitive Ability: A Comprehensive Meta-Analytic Investigation , 2017, Psychological bulletin.

[42]  I. Spence,et al.  Video Games and Spatial Cognition , 2010 .

[43]  D. Uttal,et al.  Science Education and Visuospatial Processing , 2019, Visuospatial Processing for Education in Health and Natural Sciences.

[44]  Christopher R. Engelhardt,et al.  Null Effects of Game Violence, Game Difficulty, and 2D:4D Digit Ratio on Aggressive Behavior , 2019, Psychological science.

[45]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[46]  Richard E. Mayer,et al.  The Cambridge Handbook of Multimedia Learning: Principles for Reducing Extraneous Processing in Multimedia Learning : Coherence, Signaling, Redundancy, Spatial Contiguity, and Temporal Contiguity Principles , 2005 .

[47]  Jan L. Plass,et al.  Multimedia Learning with Simulations and Microworlds , 2014 .

[48]  C. Ferguson The Good, The Bad and the Ugly: A Meta-analytic Review of Positive and Negative Effects of Violent Video Games , 2007, Psychiatric Quarterly.

[49]  T. Hammond,et al.  Video games and surgical ability: a literature review. , 2010, Journal of surgical education.

[50]  Randy L. Bell,et al.  Computer Simulations to Support Science Instruction and Learning: A critical review of the literature , 2012 .

[51]  Kirsten C. S. Adam,et al.  Improvements to visual working memory performance with practice and feedback , 2018, PloS one.

[52]  B. Hommel,et al.  Action video gaming and cognitive control: playing first person shooter games is associated with improvement in working memory but not action inhibition , 2012, Psychological Research.

[53]  Lara M. Triona,et al.  Point and Click or Grab and Heft: Comparing the Influence of Physical and Virtual Instructional Materials on Elementary School Students' Ability to Design Experiments , 2003 .

[54]  R. Mayer On the Need for Research Evidence to Guide the Design of Computer Games for Learning , 2015 .

[55]  Jeffrey Earp,et al.  An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games , 2016, Comput. Educ..

[56]  C. Sanchez Enhancing visuospatial performance through video game training to increase learning in visuospatial science domains , 2011, Psychonomic Bulletin & Review.

[57]  Roy Eagleson,et al.  Evaluation of an online three‐dimensional interactive resource for undergraduate neuroanatomy education , 2016, Anatomical sciences education.

[58]  J. Hattie,et al.  The Power of Feedback , 2007 .

[59]  Thomas Huk,et al.  Learning cell biology with close-up views or connecting lines: Evidence for the structure mapping effect , 2007, Comput. Hum. Behav..

[60]  Dylan P. Williams,et al.  Preparing Students for Practical Sessions Using Laboratory Simulation Software , 2019, Journal of Chemical Education.

[61]  Thomas Huk,et al.  Who benefits from learning with 3D models? the case of spatial ability , 2006, J. Comput. Assist. Learn..

[62]  D. Cook,et al.  Technology-enhanced simulation in emergency medicine: a systematic review and meta-analysis. , 2013, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[63]  Fred Paas,et al.  The Potential of Embodied Cognition to Improve STEAM Instructional Dynamic Visualizations , 2015 .

[64]  Jennifer J. Vogel-Walcutt,et al.  Differential impact of two types of metacognitive prompting provided during simulation-based training , 2012, Comput. Hum. Behav..

[65]  Roland Brünken,et al.  Emotional design and positive emotions in multimedia learning: An eyetracking study on the use of anthropomorphisms , 2015, Comput. Educ..

[66]  J. Castro-Alonso,et al.  Sex Differences in Visuospatial Processing , 2019, Visuospatial Processing for Education in Health and Natural Sciences.

[67]  E. Vandewater,et al.  Relation of adolescent video game play to time spent in other activities. , 2007, Archives of pediatrics & adolescent medicine.

[68]  Richard De Lisi,et al.  Computer experience and gender differences in undergraduate mental rotation performance , 1996 .

[69]  F. Paas,et al.  Computerized and Adaptable Tests to Measure Visuospatial Abilities in STEM Students , 2017 .

[70]  Cheryl I. Johnson,et al.  Designing Effective Feedback Messages in Serious Games and Simulations: A Research Review , 2017 .

[71]  James R. Brinson Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: A review of the empirical research , 2015, Comput. Educ..

[72]  S. Daniel,et al.  Can virtual reality improve anatomy education? A randomised controlled study of a computer‐generated three‐dimensional anatomical ear model , 2006, Medical education.

[73]  Jih-Hsuan Lin,et al.  Do video games exert stronger effects on aggression than film? The role of media interactivity and identification on the association of violent content and aggressive outcomes , 2013, Comput. Hum. Behav..

[74]  C. S. Green,et al.  Meta-Analysis of Action Video Game Impact on Perceptual, Attentional, and Cognitive Skills , 2018, Psychological bulletin.

[75]  Meng-Tzu Cheng,et al.  The use of serious games in science education: a review of selected empirical research from 2002 to 2013 , 2015 .

[76]  R. Mayer,et al.  Interactive Multimodal Learning Environments , 2007 .

[77]  Jennifer Wiley,et al.  The role of dynamic spatial ability in geoscience text comprehension , 2014 .

[78]  F. Paas,et al.  Embodied Cognition, Science Education, and Visuospatial Processing , 2019, Visuospatial Processing for Education in Health and Natural Sciences.

[79]  J. Pratt,et al.  Playing an Action Video Game Reduces Gender Differences in Spatial Cognition , 2007, Psychological science.

[80]  Robert Weis,et al.  Effects of Video-Game Ownership on Young Boys’ Academic and Behavioral Functioning , 2010, Psychological science.

[81]  Heather A. Priest,et al.  The Cambridge Handbook of Multimedia Learning: The Feedback Principle in Multimedia Learning , 2014 .

[82]  Laureen M. Donovan,et al.  Computer-based simulation: Effective tool or hindrance for undergraduate nursing students? , 2018, Nurse education today.

[83]  David W. Schloerb,et al.  Egocentric versus Allocentric Spatial Ability in Dentistry and Haptic Virtual Reality Training , 2013 .

[84]  J. Wiley Picture this! Effects of photographs, diagrams, animations, and sketching on learning and beliefs about learning from a geoscience text , 2018, Applied Cognitive Psychology.

[85]  Nicola J. Gibbons,et al.  The interactivity effect in multimedia learning , 2007, Comput. Educ..

[86]  G. Norman,et al.  How medical students learn spatial anatomy , 2001, The Lancet.

[87]  Matthew K. Burns,et al.  Updated Meta-Analysis of Learner Control Within Educational Technology , 2014 .

[88]  Tomasz Drabowicz,et al.  Gender and digital usage inequality among adolescents: A comparative study of 39 countries , 2014, Comput. Educ..

[89]  T. Santarius,et al.  Methodological shortcomings in the literature evaluating the role and applications of 3D training for surgical trainees , 2017, Medical teacher.

[90]  C. Ferguson Blazing Angels or Resident Evil? Can Violent Video Games be a Force for Good? , 2010 .

[91]  Fred Paas,et al.  Comparing apples and oranges? A critical look at research on learning from statics versus animations , 2016, Comput. Educ..

[92]  M. Pépin,et al.  Effect of Playing a Video Game on a Measure of Spatial Visualization , 1986, Perceptual and motor skills.

[93]  Jan T. van der Veen,et al.  The learning effects of computer simulations in science education , 2012, Comput. Educ..

[94]  Jan L. Plass,et al.  Click versus drag: User-performed tasks and the enactment effect in an interactive multimedia environment , 2014, Comput. Hum. Behav..

[95]  Fred Paas,et al.  Visuospatial tests and multimedia learning : The importance of employing relevant instruments , 2019 .

[96]  C S Green,et al.  Action-Video-Game Experience Alters the Spatial Resolution of Vision , 2007, Psychological science.

[97]  Daniel R. Montello,et al.  Spatial Reasoning With External Visualizations: What Matters Is What You See, Not Whether You Interact , 2008, Cogn. Sci..

[98]  Kasey L. Powers,et al.  Effects of video-game play on information processing: A meta-analytic investigation , 2013, Psychonomic Bulletin & Review.

[99]  Fred Paas,et al.  VAR: A Battery of Computer-Based Instruments to Measure Visuospatial Processing , 2019, Visuospatial Processing for Education in Health and Natural Sciences.

[100]  Gilles Chiniara,et al.  Twelve tips for efficient procedural simulation , 2017, Medical teacher.

[101]  R. Mayer,et al.  Role of guidance, reflection, and interactivity in an agent-based multimedia game , 2005 .

[102]  Fabienne M. Van der Kleij,et al.  Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes , 2013 .

[103]  Lara M. Triona,et al.  Hands on What? The Relative Effectiveness of Physical versus Virtual Materials in an Engineering Design Project by Middle School Children , 2007 .

[104]  C. S. Green,et al.  Action video game modifies visual selective attention , 2003, Nature.

[105]  N. Newcombe,et al.  How Important Is the Digital Divide? The Relation of Computer and Videogame Usage to Gender Differences in Mental Rotation Ability , 2005 .

[106]  Richard E. Mayer,et al.  Using multimedia for e-learning , 2017, J. Comput. Assist. Learn..

[107]  Ruth N. Schwartz,et al.  Just enough, but not too much interactivity leads to better clinical skills performance after a computer assisted learning module , 2012, Medical teacher.

[108]  Lijia Lin,et al.  Optimizing learning from animation: Examining the impact of biofeedback , 2018 .

[109]  Guido Makransky,et al.  Improving biotech education through gamified laboratory simulations , 2014, Nature Biotechnology.

[110]  Eric Jamet,et al.  Using pop-up windows to improve multimedia learning , 2006, J. Comput. Assist. Learn..

[111]  R. Moreno Decreasing Cognitive Load for Novice Students: Effects of Explanatory versus Corrective Feedback in Discovery-Based Multimedia , 2004 .

[112]  Logan Fiorella,et al.  What works and doesn't work with instructional video , 2018, Comput. Hum. Behav..

[113]  Detlef Urhahne,et al.  The Effect of Three-Dimensional Simulations on the Understanding of Chemical Structures and Their Properties , 2009 .

[114]  Mary Hegarty,et al.  Constrained interactivity for relating multiple representations in science: When virtual is better than real , 2015, Comput. Educ..