Mobile Experiences of Historical Place: A Multimodal Analysis of Emotional Engagement

This article explores how to research the opportunities for emotional engagement that mobile technologies provide for the design and enactment of learning environments. In the context of mobile technologies that foster location-based linking, we make the case for the centrality of in situ real-time observational research on how emotional engagement unfolds and for the inclusion of bodily aspects of interaction. We propose that multimodal methods offer tools for observing emotion as a central facet of person–environment interaction and provide an example of these methods put into practice for a study of emotional engagement in mobile history learning. A multimodal analysis of video data from 16 pairs of 9- to 10-year-olds learning about the World War II history of their local Common is used to illustrate how students’ emotional engagement was supported by their use of mobile devices through multimodal layering and linking of stimuli, the creation of digital artifacts, and changes in pace. These findings are significant for understanding the role of digital augmentation in fostering emotional engagement in history learning, informing how digital augmentation can be designed to effectively foster emotional engagement for learning, and providing insight into the benefits of multimodality as an analytical approach for examining emotion through bodily interaction.

[1]  R. Todd,et al.  Event-related potential measures of emotion regulation in early childhood , 2007, Neuroreport.

[2]  Yvonne Rogers,et al.  Using 'tangibles' to promote novel forms of playful learning , 2003, Interact. Comput..

[3]  Elizabeth A. Kensinger,et al.  Memory enhancement for emotional words: Are emotional words more vividly remembered than neutral words? , 2003, Memory & cognition.

[4]  J. Finn.  Withdrawing From School , 1989 .

[5]  Mar ianne Mise,et al.  Children Who Do Well in School : Individual Differences in Perceived Competence and Autonomy in Above-Average Children , 2001 .

[6]  David Kirk,et al.  Savannah: experiential learning through mobile gaming , 2004 .

[7]  P. Gagnon.  Why Study History , 1988 .

[8]  Peter Kraftl,et al.  Not just growing up, but going on: Materials, Spacings, Bodies, Situations , 2006 .

[9]  David S. Kirk,et al.  Savannah: mobile gaming and learning? , 2004, J. Comput. Assist. Learn..

[10]  H. M. Marks,et al.  Student Engagement in Instructional Activity: Patterns in the Elementary, Middle, and High School Years , 2000 .

[11]  L. Harris,et al.  A phenomenographic investigation of teacher conceptions of student engagement in learning , 2008 .

[12]  J. Banner.  Historical Thinking and Other Unnatural Acts: Charting the Future of Teaching the Past , 2003 .

[13]  Mike Sharples,et al.  A theory of learning for the mobile age: learning through conversation and exploration across contexts , 2010 .

[14]  J. Dillon,et al.  The value of outdoor learning: evidence from research in the UK and elsewhere , 2006 .

[15]  S. Wineburg,et al.  Historical Thinking and Other Unnatural Acts: Charting the Future of Teaching the Past , 2002 .

[16]  T. V. Leeuwen.  Multimodal Literacy , 2017 .

[17]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[18]  Morris Williams,et al.  Wearable computing and the geographies of urban childhood: working with children to explore the potential of new technology , 2003, IDC '03.

[19]  J. Edell,et al.  The Power of Feelings in Understanding Advertising Effects , 1987 .

[20]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[21]  K. Scherer.  Which Emotions Can be Induced by Music? What Are the Underlying Mechanisms? And How Can We Measure Them? , 2004 .

[22]  L. Schmied,et al.  The Role of Emotion in Teaching and Learning History: A Scholarship of Teaching Exploration , 2008 .

[23]  V. Morrow,et al.  Using qualitative methods to elicit young people's perspectives on their environments: some ideas for community health initiatives. , 2001, Health education research.

[24]  Jiwon Bang.  Teaching History for the Common Good , 2017 .

[25]  J. Wilkinson.  Feeling an emotion , 1998 .

[26]  Carmelo Ardito,et al.  Enabling Interactive Exploration of Cultural Heritage: An Experience of Designing Systems for Mobile Devices , 2009 .

[27]  P. Gomez,et al.  Relationships between musical structure and psychophysiological measures of emotion. , 2007, Emotion.

[28]  J. Gross.  The Emerging Field of Emotion Regulation: An Integrative Review , 1998 .

[29]  Wilfried Admiraal,et al.  Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game , 2009, J. Comput. Assist. Learn..

[30]  J. Rosiek,et al.  Emotional Scaffolding , 2003 .

[31]  Christian Heath,et al.  Handbook of Multimodal Analysis , 2009 .

[32]  Pieter Desmet,et al.  Measuring Emotion: Development and Application of an Instrument to Measure Emotional Responses to Products , 2005, Funology.

[33]  Anastasia Efklides,et al.  Effects of mood on students' metacognitive experiences , 2005 .

[34]  Yvonne Rogers,et al.  Ambient wood: designing new forms of digital augmentation for learning outdoors , 2004, IDC '04.

[35]  M. Hidalgo,et al.  Place attachment: Conceptual and empirical questions , 2001 .

[36]  O. L. Davis,et al.  Historical Empathy and Perspective Taking in the Social Studies , 2001 .

[37]  E. Gonida,et al.  Students' achievement goal orientations and their behavioral and emotional engagement: Co-examining the role of perceived school goal structures and parent goals during adolescence , 2009 .

[38]  Sue J. Waite,et al.  ‘Memories are made of this’: some reflections on outdoor learning and recall , 2007 .

[39]  M. Carrasco,et al.  PSYCHOLOGICAL SCIENCE Research Article Emotion Facilitates Perception and Potentiates the Perceptual Benefits of Attention , 2022 .

[40]  Pat Wolfe,et al.  The role of meaning and emotion in learning , 2006 .

[41]  Kurt Squire,et al.  Augmented Reality Simulations on Handheld Computers , 2007 .

[42]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[43]  Stuart Foster,et al.  The Role of Empathy in the Development of Historical Understanding , 1998 .

[44]  J. Gross,et al.  The tie that binds? Coherence among emotion experience, behavior, and physiology. , 2005, Emotion.

[45]  M. Bradley,et al.  Measuring emotion: Behavior, feeling, and physiology , 2000 .

[46]  Katherine B. Martin,et al.  Facial Action Coding System , 2015 .

[47]  Elizabeth A. Linnenbrink,et al.  Achievement Goal Theory and Affect: An Asymmetrical Bidirectional Model , 2002 .

[48]  J. Salmon,et al.  Children's perceptions of their home and neighborhood environments, and their association with objectively measured physical activity: a qualitative and quantitative study. , 2004, Health education research.

[49]  Lars Erik Holmquist,et al.  Tools for Students Doing Mobile Fieldwork , 2008, Fifth IEEE International Conference on Wireless, Mobile, and Ubiquitous Technology in Education (wmute 2008).

[50]  S. Volk.  How the Air Felt on My Cheeks: Using Avatars to Access History. , 2013 .

[51]  C. Goodwin.  Action and embodiment within situated human interaction , 2000 .

[52]  M. F. Orellana.  Space and place in an Urban Landscape: Learning from children's views of their social worlds , 1999 .

[53]  G. Clore,et al.  Feelings and phenomenal experiences , 1996 .

[54]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[55]  Carey Jewitt,et al.  Multimodal methods for researching digital technologies , 2013 .

[56]  Christine Milligan,et al.  Embodying emotion sensing space: introducing emotional geographies , 2004 .

[57]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[58]  G. Bonanno,et al.  Brief Report The coherence of emotion systems: Comparing “on‐line” measures of appraisal and facial expressions, and self‐report , 2004 .

[59]  Yvonne Rogers,et al.  Let's get physical: The learning benefits of interacting in digitally augmented physical spaces , 2004, Comput. Educ..

[60]  Dionne I. Cross,et al.  Reflections on Investigating Emotion in Educational Activity Settings , 2006 .