Collaborative Embodied Learning in Mixed Reality Motion-Capture Environments: Two Science Studies

These 2 studies investigate the extent to which an Embodied Mixed Reality Learning Environment (EMRELE) can enhance science learning compared to regular classroom instruction. Mixed reality means that physical tangible and digital components were present. The content for the EMRELE required that students map abstract concepts and relations onto their gestures and movements so that the concepts would become grounded in embodied action. The studies compare an immersive, highly interactive learning platform that uses a motion-capture system to track students’ gestures and locomotion as they kinesthetically learn with a quality classroom experience (teacher and content were held constant). Two science studies are presented: chemistry titration and disease transmission. In the counterbalanced design 1 group received the EMRELE intervention, while the other group received regular instruction; after 3 days and a midtest, the interventions switched. Each study lasted for 6 days total, with 3 test points: pretest, midtest, and posttest. Analyses revealed that placement in the embodied EMRELE condition consistently led to greater learning gains (effect sizes ranged from 0.53 to 1.93), compared to regular instruction (effect sizes ranged from 0.09 to 0.37). Order of intervention did not affect the final outcomes at posttest. These results are discussed in relation to a new taxonomy of embodiment in educational settings. We hypothesize that the positive results are due to the embodiment designed into the lessons and the high degree of collaboration engendered by the co-located EMRELE.

[1]  David Birchfield,et al.  Learning metaphor through mixed-reality game design and game play , 2008, Sandbox '08.

[2]  Sharon E. Smaldino,et al.  Instructional Technology and Media for Learning , 2013 .

[3]  Willis F. Overton,et al.  Developmental Perspectives on Embodiment and Consciousness , 2008 .

[4]  J. Wilder The Origins of Intelligence in Children , 1954 .

[5]  Lara M. Triona,et al.  Hands on What? The Relative Effectiveness of Physical versus Virtual Materials in an Engineering Design Project by Middle School Children , 2007 .

[6]  Michio Kaku Physics of the Future: How Science Will Shape Human Destiny and Our Daily Lives by the Year 2100 , 2012 .

[7]  Jan L. Plass,et al.  Design factors for educationally effective animations and simulations , 2009, J. Comput. High. Educ..

[8]  J. Piaget,et al.  The Origins of Intelligence in Children , 1971 .

[9]  F. Craik,et al.  Novelty and familiarity activations in PET studies of memory encoding and retrieval. , 1996, Cerebral cortex.

[10]  Kelly S. Shapley,et al.  Evaluation of the Texas Technology Immersion Pilot: Final Outcomes for a Four-Year Study (2004-05 to 2007-08). , 2009 .

[11]  Garry Falloon,et al.  Using avatars and virtual environments in learning: What do they have to offer? , 2010, Br. J. Educ. Technol..

[12]  Mina C. Johnson-Glenberg,et al.  Web-Based Training of Metacognitive Strategies for Text Comprehension: Focus on Poor Comprehenders , 2005 .

[13]  David Birchfield,et al.  Embodied Games, Next Gen Interfaces, and Assessment of High School Physics , 2009 .

[14]  G. Lakoff,et al.  Where mathematics comes from : how the embodied mind brings mathematics into being , 2002 .

[15]  C. Heyes,et al.  Science Perspectives on Psychological Mirror Neuron Forum on Behalf Of: Association for Psychological Science , 2022 .

[16]  David Birchfield,et al.  Teaching and Learning in the Mixed-Reality Science Classroom , 2009 .

[17]  Allison Winters,et al.  Emotion, Embodiment, and Mirror Neurons in Dance/Movement Therapy: A Connection Across Disciplines , 2008 .

[18]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[19]  Susan Goldin-Meadow,et al.  Gesturing makes memories that last. , 2010, Journal of memory and language.

[20]  Michelene T. H. Chi,et al.  Active-Constructive-Interactive: A Conceptual Framework for Differentiating Learning Activities , 2009, Top. Cogn. Sci..

[21]  David Birchfield,et al.  Earth science learning in SMALLab: A design experiment for mixed reality , 2009, Int. J. Comput. Support. Collab. Learn..

[22]  Friedemann Pulvermüller,et al.  Brain mechanisms linking language and action , 2005, Nature Reviews Neuroscience.

[23]  David W. Johnson,et al.  Learning together and alone: Cooperation, competition, and individualization , 1975 .

[24]  Hee-Sun Lee,et al.  Making authentic science accessible to students , 2003 .

[25]  Mina C. Johnson-Glenberg,et al.  Web-based Reading Comprehension Instruction: Three Studies of 3D-readers , 2007 .

[26]  Bertram C. Bruce Computers and the collaborative experience of learning , 1995 .

[27]  Michael F. Young,et al.  Our Princess Is in Another Castle , 2012 .

[28]  James Paul Gee,et al.  Good Video Games + Good Learning: collected essays on video games, learning and literacy , 2013 .

[29]  Penelope M Sanderson,et al.  Monitoring with Head-Mounted Displays in General Anesthesia: A Clinical Evaluation in the Operating Room , 2010, Anesthesia and analgesia.

[30]  Arne D. Ekstrom,et al.  Single-Neuron Responses in Humans during Execution and Observation of Actions , 2010, Current Biology.

[31]  Sadhana Puntambekar,et al.  The effects of physical and virtual manipulatives on students' conceptual learning about pulleys , 2010, ICLS.

[32]  K. VanLehn,et al.  Scaffolding Deep Comprehension Strategies Through Point&Query, AutoTutor, and iSTART , 2005 .

[33]  Jean-François Rouet,et al.  Understanding multimedia documents - an introduction , 2008 .

[34]  樊希强,et al.  Cooperative Learning , 2020, Teaching High School Physics.

[35]  Hubert D. Zimmer,et al.  Motor programs and their relation to semantic memory. , 1985 .

[36]  Arthur C. Graesser,et al.  Natural Language Tutoring: A comparison of human tutors, computer tutors, and text , 2007 .

[37]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[38]  David Birchfield,et al.  A Next Gen Interface for Embodied Learning: SMALLab and the Geological Layer Cake , 2010, Int. J. Gaming Comput. Mediat. Simulations.

[39]  David Birchfield,et al.  SMALLab: virtual geology studies using embodied learning with motion, sound, and graphics , 2009 .

[40]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[41]  Arben Asllani,et al.  Sharing knowledge with conversational technologies: web logs versus discussion boards , 2008, Int. J. Inf. Technol. Manag..

[42]  William R. Penuel,et al.  Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort , 2007 .

[43]  K. VanLehn The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[44]  David W. Johnson,et al.  Cooperation and Competition: Theory and Research , 1989 .

[45]  Robb Lindgren,et al.  Supporting children's learning with body-based metaphors in a mixed reality environment , 2011, IDC.

[46]  Travel broadens the mind--you bet! , 2001, Journal of tissue viability.

[47]  Gillian Smith,et al.  Affordances and Constraints of Computers in Science Education , 1996 .

[48]  F. Fischer,et al.  A framework to analyze argumentative knowledge construction in computer-supported collaborative learning , 2006, Comput. Educ..

[49]  Dennis Proffitt,et al.  Quantifying immersion in virtual reality , 1997, SIGGRAPH.

[50]  David W. Johnson,et al.  An Educational Psychology Success Story: Social Interdependence Theory and Cooperative Learning , 2009 .

[51]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[52]  A. Glenberg Embodiment as a unifying perspective for psychology. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[53]  David W. Johnson,et al.  Cooperation and Competition , 2015 .

[54]  Helga Noice,et al.  What Studies of Actors and Acting Can Tell Us About Memory and Cognitive Functioning , 2006 .

[55]  L. Fadiga,et al.  Active perception: sensorimotor circuits as a cortical basis for language , 2010, Nature Reviews Neuroscience.

[56]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[57]  L. Vygotsky Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[58]  V. Shute Focus on Formative Feedback , 2008 .

[59]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[60]  Frank C. Leeming,et al.  Deep Learning and Emotion in Serious Games , 2009 .

[61]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[62]  Ralf Steinmetz,et al.  Designing Collaborative Multiplayer Serious Games for Collaborative Learning - Escape from Wilson Island - A Multiplayer 3D Serious Game for Collaborative Learning in Teams , 2012, CSEDU.

[63]  Priya K. Nihalani,et al.  When Feedback Harms and Collaboration Helps in Computer Simulation Environments: An Expertise Reversal Effect , 2011 .

[64]  Luigi Cattaneo,et al.  Use-induced motor plasticity affects the processing of abstract and concrete language , 2008, Current Biology.

[65]  Lesya M. Hassall,et al.  An Examination of a Theory of Embodied Social Presence in Virtual Worlds , 2011, Decis. Sci..

[66]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.

[67]  Daniel L. Schwartz,et al.  Shuttling Between Depictive Models and Abstract Rules: Induction and Fallback , 1996, Cogn. Sci..

[68]  Mitchell J. Nathan,et al.  Embodiment in Mathematics Teaching and Learning: Evidence From Learners' and Teachers' Gestures , 2012 .

[69]  Susan Goldin-Meadow,et al.  Learning through gesture. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[70]  Autumn B. Hostetter,et al.  Visible embodiment: Gestures as simulated action , 2008, Psychonomic bulletin & review.

[71]  W. Schnotz,et al.  A Reconsideration of Cognitive Load Theory , 2007 .