Click versus drag: User-performed tasks and the enactment effect in an interactive multimedia environment

Does learner performance of specific tasks in an interactive multimedia environment affect learning outcomes? Within a multimedia environment, users may engage in a range of actions, or interactive tasks, from tapping a keyboard to executing large motor movements. To investigate the impact of particular performance tasks on learning, we first introduce an approach to classifying interactive multimedia tasks as enactive, iconic or symbolic. We then describe a study in which university students (N=112) used a computer program that presented a series of action phrases in 4 different conditions, each condition requiring performance of a different task: listen (audio only), look (audio with static graphic), click (audio with animation triggered by a click), or drag (audio with click-and-drag graphic). Participants were tested on free recall and recognition of phrases immediately after treatment and again after 3weeks. At immediate testing, recall was best for drag (iconic) items, followed by click (symbolic), look, and listen items, in that order, with significant differences between each pair of conditions. For immediate recognition, as well as for delayed free recall and delayed recognition, mean scores followed the same pattern, with some variations in significance. Results support our proposed classification of interactive behaviors, extend previous findings on the enactment effect into a computer environment, and suggest the importance of considering the design of interactive tasks in the development of multimedia learning materials.

[1]  Katherine Isbister,et al.  Waggling the Form Baton: Analyzing Body-Movement-Based Design Patterns in Nintendo Wii Games, Toward Innovation of New Possibilities for Social and Emotional Experience , 2011, Whole Body Interaction.

[2]  Jennifer Lawrence,et al.  Communication in the Age of Virtual Reality. , 1997 .

[3]  Dor Abrahamson,et al.  Embodied Artifacts and Conceptual Performances , 2012, ICLS.

[4]  Hubert D. Zimmer,et al.  Sensory factors in memory for subject-performed tasks , 1997 .

[5]  W. Michael Reed,et al.  The Effect of Interactivity with a Music Video Game on Second Language Vocabulary Recall. , 2010 .

[6]  Richard A. Schwier,et al.  Interactive Multimedia Instruction , 1993 .

[7]  A. Glenberg,et al.  Enhancing comprehension in small reading groups using a manipulation strategy , 2007 .

[8]  S. Goldin-Meadow,et al.  Pointing sets the stage for learning language--and creating language. , 2007, Child development.

[9]  J. Bruner.  Toward a Theory of Instruction , 1966 .

[10]  Vladimir M. Sloutsky,et al.  From Perceptual Categories to Concepts: What Develops? , 2010, Cogn. Sci..

[11]  Jana Rambusch.  Riding a bike in Paperboy : A case study of embodied humancomputer game interaction , 2007 .

[12]  William Haseman,et al.  An Empirical Investigation of the Influences of the Degree of Interactivity on User-Outcomes in a Multimedia Environment , 2002, Inf. Resour. Manag. J..

[13]  Eunjoon Rachel Um,et al.  Emotional design in multimedia learning. , 2012 .

[14]  J. Engelkamp,et al.  Routes to actions and their efficacy for remembering. , 1996, Memory.

[15]  D. Krathwohl.  A Revision of Bloom's Taxonomy: An Overview , 2002 .

[16]  Hubert D. Zimmer,et al.  Recall and recognition of self-performed acts , 1989 .

[17]  Ruth N. Schwartz,et al.  Effects of pacing and cognitive style across dynamic and non-dynamic representations , 2011, Comput. Educ..

[18]  Roland Brünken,et al.  Cognitive Load Theory: THEORY , 2010 .

[19]  J. Bruner.  Studies in cognitive growth : a collaboration at the Center for Cognitive Studies , 1966 .

[20]  Patrick Péruch,et al.  Active versus passive learning and testing in a complex outside built environment , 2004, Cognitive Processing.

[21]  Dorothy M. Chun,et al.  Effects of Multimedia Annotations on Vocabulary Acquisition , 1996 .

[22]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[23]  Michael P. Kaschak,et al.  Activity and Imagined Activity Can Enhance Young Children's Reading Comprehension. , 2004 .

[24]  A. Jennekens‐Schinkel,et al.  The human memory: A multi-modal approach revised translation of ‘Das menschliche Gedächtnis’ (1991), by J. Engelkamp and H.D. Zimmer, 1994, Hogrefe and Huber, Toronto, 532 pages US$ 54.00, DM 89.00, ISBN 0-88937-116-4 , 1995, Clinical Neurology and Neurosurgery.

[25]  J. Dewey.  Democracy and education : an introduction to philosophy of education / John Dewey , 1916 .

[26]  Abigail Sellen,et al.  A comparison of input devices in element pointing and dragging tasks , 1991, CHI.

[27]  Jörg Zumbach,et al.  Influence of Passive versus Active Information Access to Hypertextual Information Resources on Cognitive and Emotional Parameters , 2001 .

[28]  Nicola J. Gibbons,et al.  The interactivity effect in multimedia learning , 2007, Comput. Educ..

[29]  Kori Inkpen Quinn,et al.  Drag-and-drop versus point-and-click mouse interaction styles for children , 2001, TCHI.

[30]  Mireille Betrancourt,et al.  The Cambridge Handbook of Multimedia Learning: The Animation and Interactivity Principles in Multimedia Learning , 2005 .

[31]  Maria Montessori,et al.  Dr. Montessori's Own Handbook , 1914 .

[32]  S. Mackenzie,et al.  A comparison of input device in elemental pointing and dragging task , 1991, CHI 1991.

[33]  S. Goldin-Meadow,et al.  Gesturing makes learning last , 2008, Cognition.

[34]  R. Moreno.  Decreasing Cognitive Load for Novice Students: Effects of Explanatory versus Corrective Feedback in Discovery-Based Multimedia , 2004 .

[35]  Jan L. Plass,et al.  Interactivity in Multimedia Learning , 2011 .

[36]  J. Engelkamp.  Memory for actions , 1998 .

[37]  W. Schnotz.  Integrated Model of Text and Picture Comprehension , 2021, The Cambridge Handbook of Multimedia Learning.

[38]  H L Roediger,et al.  Imagination inflation for action events: Repeated imaginings lead to illusory recollections , 1998, Memory & cognition.

[39]  M. Wittrock.  Generative Learning Processes of the Brain , 1992 .

[40]  Slava Kalyuga,et al.  Enhancing Instructional Efficiency of Interactive E-learning Environments: A Cognitive Load Perspective , 2007 .

[41]  Lars-Göran Nilsson,et al.  Recall of enacted and nonenacted instructions compared: forgetting functions , 1989 .

[42]  Friedrich Fröbel.  Friedrich Froebel's Pedagogics of the Kindergarten: Or, His Ideas Concerning the Play and Playthings of the Child , 2010 .

[43]  Oliver Quiring,et al.  Interactivity: A review of the concept and a framework for analysis , 2008 .

[44]  M. Bannert,et al.  Construction and interference in learning from multiple representation , 2003 .

[45]  A. Clark.  Being There: Putting Brain, Body, and World Together Again , 1996 .

[46]  Béatrice S. Hasler,et al.  Learner Control, Cognitive Load and Instructional Animation , 2007 .

[47]  Johannes Engelkamp,et al.  Lexical, conceptual and motor information in memory for action phrases: a multi-system account. , 2003, Acta psychologica.

[48]  Regan L. Mandryk,et al.  Understanding children's collaborative interactions in shared environments , 2003, J. Comput. Assist. Learn..

[49]  Afke Donker,et al.  Young children's ability to use a computer mouse , 2007, Comput. Educ..

[50]  Björn B. de Koning,et al.  Facilitating Understanding of Movements in Dynamic Visualizations: an Embodied Perspective , 2011 .

[51]  A. Paivio,et al.  Dual coding theory and education , 1991 .

[52]  Magda Osman,et al.  Seeing is as Good as Doing , 2008, J. Probl. Solving.

[53]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[54]  Fred Paas,et al.  Instructional efficiency of animation: effects of interactivity through mental reconstruction of static key frames , 2007 .

[55]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[56]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[57]  Rod Sims,et al.  Interactivity: A Forgotten Art? , 1997 .

[58]  Kyle L. Peck.  The Design Development and Evaluation of Instructional Software , 1988 .

[59]  R. Mayer,et al.  Interactive Multimodal Learning Environments , 2007 .

[60]  John Brown,et al.  Recall and Recognition , 1976 .

[61]  Samsup Jo,et al.  The Effect of Web Characteristics on Relationship Building , 2003 .

[62]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[63]  S. Schwan,et al.  The cognitive benefits of interactive videos: learning to tie nautical knots , 2004 .

[64]  Gregor Kennedy,et al.  Promoting Cognition in Multimedia Interactivity Research. , 2004 .

[65]  Johann Heinrich Pestalozzi.  Letters on Early Education: Addressed to J. P, Greaves, Esq. , 2010 .

[66]  Alexander Renkl,et al.  Interactive Example-Based Learning Environments: Using Interactive Elements to Encourage Effective Processing of Worked Examples , 2007 .

[67]  Dorothy M. Chun,et al.  SUPPORTING VISUAL AND VERBAL LEARNING PREFERENCES IN A SECOND LANGUAGE MULTIMEDIA LEARNING ENVIRONMENT , 1998 .

[68]  Elizabeth O. Hayward,et al.  Emotional design in multimedia learning: Effects of shape and color on affect and learning , 2014 .

[69]  David A. Lieberman.  Human Learning and Memory , 2012 .

[70]  J Engelkamp,et al.  Item and order information in subject-performed tasks and experimenter-performed tasks. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[71]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[72]  R. Mayer,et al.  Role of guidance, reflection, and interactivity in an agent-based multimedia game , 2005 .

[73]  Xiaojin Zhu,et al.  Improving early reading comprehension using embodied CAI , 2011 .

[74]  R. Mayer,et al.  When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? , 2001 .

[75]  Sam Reid,et al.  A Study of Educational Simulations Part 1 - Engagement and Learning , 2008 .

[76]  D. Leutner,et al.  Instructional animation versus static pictures: A meta-analysis , 2007 .

[77]  S. Mewaldt,et al.  The human memory: A multi-modal approach Johannes Engelkamp and Hubert D. Zimmer. Hogrefe & Huber Publishers, Seattle and Toronto, 1994; ISBN 0-88937-116-4; US$54.00 , 1996, Journal of Chemical Neuroanatomy.

[78]  Jan L. Plass,et al.  Interactivity in multimedia learning: An integrated model , 2010, Comput. Hum. Behav..

[79]  Marina Basu.  The Embodied Mind: Cognitive Science and Human Experience , 2004 .

[80]  Wolfgang Schnotz,et al.  The Cambridge Handbook of Multimedia Learning: An Integrated Model of Text and Picture Comprehension , 2005 .

[81]  EvansChris,et al.  The interactivity effect in multimedia learning , 2007 .

[82]  Slava Kalyuga,et al.  Design Factors for Effective Science Simulations: Representation of Information , 2009, Int. J. Gaming Comput. Mediat. Simulations.

[83]  Sylvia Stracke,et al.  Comparative evaluation of multimedia driven, interactive, and case-based teaching in heart surgery. , 2006, The Annals of thoracic surgery.

[84]  Sinan Olkun,et al.  Why Interactivity Works: Interactive Priming of Mental Rotation , 2005 .

[85]  Dor Abrahamson,et al.  Toward an embodied-interaction design framework for mathematical concepts , 2011, IDC.