Emotion and Body-Based Games: Overview and Opportunities

In this chapter we examine research and theory concerning body movement as a means for expressing emotion, and techniques for recognizing expressions of emotion in the context of games. We discuss body movement as a means for biasing emotional experience and encouraging bonding in social interaction. Finally, we discuss gaps and opportunities for future research. Promising directions include broadening the scope of body-based games and emotion to take proprioception into account, as well as other less explored body channels such as muscle activation and action-related sound.

[1]  Elena Márquez Segura,et al.  Enabling Co-Located Physical Social Play: A Framework for Design and Evaluation , 2015, Game User Experience Evaluation.

[2]  Katherine Isbister,et al.  Wriggle: an exploration of emotional and social effects of movement , 2011, CHI Extended Abstracts.

[3]  Catherine Pelachaud,et al.  Relevant body cues for the classification of emotional body expression in daily actions , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[4]  Stuart Reeves,et al.  Designing Interfaces in Public Settings - Understanding the Role of the Spectator in Human-Computer Interaction , 2011, Human-Computer Interaction Series.

[5]  N. Dael,et al.  Measuring body movement: Current and future directions in proxemics and kinesics. , 2015 .

[6]  Nadia Bianchi-Berthouze,et al.  Does Movement Recognition Precision Affect the Player Experience in Exertion Games? , 2011, INTETAIN.

[7]  Richard E. Petty,et al.  Body posture effects on self‐evaluation: A self‐validation approach , 2009 .

[8]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .

[9]  Maja Pantic,et al.  The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset , 2016, IEEE Transactions on Affective Computing.

[10]  Janne Paavilainen,et al.  The many faces of sociability and social play in games , 2009, MindTrek '09.

[11]  Ana Tajadura-Jiménez,et al.  As Light as your Footsteps: Altering Walking Sounds to Change Perceived Body Weight, Emotional State and Gait , 2015, CHI.

[12]  Andrea Kleinsmith,et al.  A categorical approach to affective gesture recognition , 2003, Connect. Sci..

[13]  Kristina Höök,et al.  On being supple: in search of rigor without rigidity in meeting new design and evaluation challenges for HCI practitioners , 2009, CHI.

[14]  Jens Heydekorn,et al.  Click or Strike: Realistic versus Standard Game Controls in Violent Video Games and Their Effects on Aggression , 2010, ICEC.

[15]  Andrea Kleinsmith,et al.  Form as a Cue in the Automatic Recognition of Non-acted Affective Body Expressions , 2011, ACII.

[16]  Henrik Hautop Lund,et al.  Playware technology for physically activating play , 2005, Artificial Life and Robotics.

[17]  Michael J. Richardson,et al.  Social Connection Through Joint Action and Interpersonal Coordination , 2009, Top. Cogn. Sci..

[18]  D. Casasanto,et al.  Motor action and emotional memory , 2010, Cognition.

[19]  Bart Simon,et al.  Wii are Out of Control: Bodies, Game Screens and the Production of Gestural Excess , 2009 .

[20]  D. Keltner,et al.  The communication of emotion via touch. , 2009, Emotion.

[21]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[22]  A. Atkinson,et al.  Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures , 2007, Cognition.

[23]  Anthony Steed,et al.  Automatic Recognition of Non-Acted Affective Postures , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Gavin Wood,et al.  i-dentity: innominate movement representation as engaging game element , 2014, CHI.

[25]  R. Laban,et al.  The mastery of movement , 1950 .

[26]  Nicolai Marquardt,et al.  Bi-Modal Detection of Painful Reaching for Chronic Pain Rehabilitation Systems , 2014, ICMI.

[27]  C. Gotay,et al.  Physical posture: Could it have regulatory or feedback effects on motivation and emotion? , 1982 .

[28]  Nadia Bianchi-Berthouze,et al.  Understanding the Role of Body Movement in Player Engagement , 2012, Hum. Comput. Interact..

[29]  Nadia Bianchi-Berthouze,et al.  Tactile perceptions of digital textiles: a design research approach , 2013, CHI.

[30]  Elena Márquez Segura,et al.  The design space of body games: technological, physical, and social design , 2013, CHI.

[31]  William Curran,et al.  Perception and Automatic Recognition of Laughter from Whole-Body Motion: Continuous and Categorical Perspectives , 2015, IEEE Transactions on Affective Computing.

[32]  Mark C. Coulson.  Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence , 2004 .

[33]  Ana Tajadura-Jiménez,et al.  Motivating people with chronic pain to do physical activity: opportunities for technology design , 2014, CHI.

[34]  M. Tsakiris,et al.  Action sounds update the mental representation of arm dimension: contributions of kinaesthesia and agency , 2015, Front. Psychol..

[35]  Katherine Isbister,et al.  Movement-based game guidelines , 2014, CHI.

[36]  Katherine Isbister,et al.  Pixel Motion: A surveillance camera-enabled public digital game , 2014, FDG.

[37]  Andrea Kleinsmith,et al.  Cross-cultural differences in recognizing affect from body posture , 2006, Interact. Comput..

[38]  Radoslaw Niewiadomski,et al.  Laugh When You're Winning , 2013, eNTERFACE.

[39]  Mohamed R. Amer,et al.  The Tower Game Dataset: A multimodal dataset for analyzing social interaction predicates , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[40]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[41]  H. Wallbott.  Bodily expression of emotion , 1998 .

[42]  A. Damasio.  Descartes' error: emotion, reason, and the human brain. avon books , 1994 .

[43]  William Curran,et al.  Laughter Type Recognition from Whole Body Motion , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[44]  P. Kleinginna,et al.  A categorized list of emotion definitions, with suggestions for a consensual definition , 1981 .

[45]  Andrea Kleinsmith,et al.  Recognizing Affective Dimensions from Body Posture , 2007, ACII.

[46]  Katie Salen,et al.  Rules of play: game design fundamentals , 2003 .

[47]  Federico Fontana,et al.  Expressive sonification of footstep sounds , 2010 .

[48]  E. Hall,et al.  Proxemics [and Comments and Replies] , 1968, Current Anthropology.

[49]  Yiorgos Chrysanthou,et al.  Affect Recognition during Active Game Playing based on Posture Skeleton Data , 2013, GRAPP/IVAPP.

[50]  Andreas Aristidou,et al.  Emotion Recognition for Exergames using Laban Movement Analysis , 2013, MIG.

[51]  Nadia Bianchi-Berthouze,et al.  Does Body Movement Engage You More in Digital Game Play? and Why? , 2007, ACII.

[52]  Philip Hingston,et al.  Considerations for the design of exergames , 2007, GRAPHITE '07.

[53]  Andrea Kleinsmith,et al.  Grounding Affective Dimensions into Posture Features , 2005, ACII.

[54]  Florian Mueller,et al.  Musical embrace: socially awkward interactions through physical proximity to drive digital play , 2013, CHI Extended Abstracts.

[55]  Radoslaw Niewiadomski,et al.  Rhythmic Body Movements of Laughter , 2014, ICMI.

[56]  Anton Nijholt,et al.  Movement-Based Sports Video Games: Investigating Motivation and Gaming Experience , 2009, Entertain. Comput..

[57]  Nadia Bianchi-Berthouze,et al.  Stirring up experience through movement in game play: effects on engagement and social behaviour , 2008, CHI.

[58]  R. Petty,et al.  lhe Effects of Overt Head Movements on Persuasion: Compatibility and incompatibility of Responses , 1980 .

[59]  Nicolai Marquardt,et al.  Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[60]  John T. Cacioppo,et al.  Emotional Contagion: Acknowledgments , 1993 .

[61]  Markus Lappe,et al.  The role of spatial and temporal information in biological motion perception , 2008, Advances in cognitive psychology.

[62]  M. Tsakiris.  My body in the brain: A neurocognitive model of body-ownership , 2010, Neuropsychologia.

[63]  Maurizio Mancini,et al.  Gesture mimicry in expression of laughter , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[64]  Yiorgos Chrysanthou,et al.  The Next Big Thing Automatic Emotion Recognition Based on Body Movement Analysis A Survey , 2014 .

[65]  B. Gelder,et al.  The Body Action Coding System I: muscle activations during the perception and expression of emotion. , 2014 .

[66]  Antonio Camurri,et al.  Analysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library , 2003, Gesture Workshop.

[67]  Catherine Pelachaud,et al.  Multi-level classification of emotional body expression , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[68]  J. Cacioppo,et al.  Rudimentary determinants of attitudes. II: Arm flexion and extension have differential effects on attitudes. , 1993, Journal of personality and social psychology.

[69]  P. Ekman,et al.  Head and body cues in the judgment of emotion: a reformulation. , 1967, Perceptual and motor skills.

[70]  Radoslaw Niewiadomski,et al.  How is your laugh today? , 2014, CHI Extended Abstracts.

[71]  Y. Trope,et al.  Body Cues, Not Facial Expressions, Discriminate Between Intense Positive and Negative Emotions , 2012, Science.

[72]  Michelle Karg,et al.  Recognition of Affect Based on Gait Patterns , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[73]  Katherine Isbister,et al.  How to stop being a buzzkill: designing yamove!, a mobile tech mash-up to truly augment social play , 2012, Mobile HCI.

[74]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[75]  Georgios N. Yannakakis,et al.  Entertainment modeling through physiology in physical play , 2008, Int. J. Hum. Comput. Stud..

[76]  Florian Mueller,et al.  Understanding a Socially Awkward Digital Play Journey , 2013, DiGRA Conference.

[77]  Radoslaw Niewiadomski,et al.  Automated Laughter Detection From Full-Body Movements , 2016, IEEE Transactions on Human-Machine Systems.

[78]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[79]  J. Huizinga.  Homo Ludens: A Study of the Play-Element in Culture , 1938 .

[80]  Claire L. Roether,et al.  Critical features for the perception of emotion from gait. , 2009, Journal of vision.

[81]  Nadia Bianchi-Berthouze,et al.  Continuous Recognition of Player's Affective Body Expression as Dynamic Quality of Aesthetic Experience , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[82]  M. S. Hane Aung,et al.  Automatic recognition of fear-avoidance behavior in chronic pain physical rehabilitation , 2014, PervasiveHealth.

[83]  B. Gelder,et al.  Why bodies? Twelve reasons for including bodily expressions in affective neuroscience , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[84]  Jennifer G. Sheridan,et al.  Designing sports: a framework for exertion games , 2011, CHI.

[85]  Piercarlo Valdesolo,et al.  Synchrony and the social tuning of compassion. , 2011, Emotion.

[86]  Robby van Delden,et al.  Socially Aware Interactive Playgrounds , 2013, IEEE Pervasive Computing.

[87]  Katherine Isbister,et al.  Guidelines for the Design of Movement-Based Games and Their Relevance to HCI , 2015, Hum. Comput. Interact..

[88]  Nasir D. Memon,et al.  Biometric-rich gestures: a novel approach to authentication on multi-touch devices , 2012, CHI.

[89]  Amy J. C. Cuddy,et al.  Power Posing , 2010, Psychological science.