Real-time Automatic Emotion Recognition from Body Gestures

Although psychological research indicates that bodily expressions convey important affective information, to date research in emotion recognition focused mainly on facial expression or voice analysis. In this paper we propose an approach to realtime automatic emotion recognition from body movements. A set of postural, kinematic, and geometrical features are extracted from sequences 3D skeletons and fed to a multi-class SVM classifier. The proposed method has been assessed on data acquired through two different systems: a professionalgrade optical motion capture system, and Microsoft Kinect. The system has been assessed on a "six emotions" recognition problem, and using a leave-one-subject-out cross validation strategy, reached an overall recognition rate of 61.3% which is very close to the recognition rate of 61.9% obtained by human observers. To provide further testing of the system, two games were developed, where one or two users have to interact to understand and express emotions with their body.

[1]  Peter F. Driessen,et al.  Gesture-Based Affective Computing on Motion Capture Data , 2005, ACII.

[2]  Peter Bull.  Posture and Gesture , 2016 .

[3]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[4]  Antonio Camurri,et al.  Toward a Minimal Representation of Affective Gestures , 2011, IEEE Transactions on Affective Computing.

[5]  P. Ekman.  Differential communication of affect by head and body cues. , 1965, Journal of personality and social psychology.

[6]  Kostas Karpouzis,et al.  Emotion Analysis in Man-Machine Interaction Systems , 2004, MLMI.

[7]  S. Baron-Cohen,et al.  Enhancing Emotion Recognition in Children with Autism Spectrum Conditions: An Intervention Using Animated Vehicles with Real Emotional Faces , 2010, Journal of autism and developmental disorders.

[8]  S. Baron-Cohen,et al.  The Cambridge Mindreading (CAM) Face-Voice Battery: Testing Complex Emotion Recognition in Adults with and without Asperger Syndrome , 2006, Journal of autism and developmental disorders.

[9]  J. Cunningham,et al.  Children's decoding of emotion in expressive body movement: the development of cue attunement. , 1998, Developmental psychology.

[10]  Mark C. Coulson.  Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence , 2004 .

[11]  Nicoletta Noceti,et al.  Learning Behavioral Patterns of Time Series for Video-Surveillance , 2011 .

[12]  Randolph R. Cornelius.  THEORETICAL APPROACHES TO EMOTION , 2000 .

[13]  Ashish Kapoor,et al.  Automatic prediction of frustration , 2007, Int. J. Hum. Comput. Stud..

[14]  Annett Baier,et al.  Modern Educational Dance , 2016 .

[15]  K. Scherer,et al.  Bodily expression of emotion , 2009 .

[16]  M. D. Meijer.  The contribution of general features of body movement to the attribution of emotions , 1989 .

[17]  Nicoletta Noceti,et al.  Learning common behaviors from large sets of unlabeled temporal series , 2012, Image Vis. Comput..

[18]  Alon Orlitsky,et al.  Combined binary classifiers with applications to speech recognition , 2002, INTERSPEECH.

[19]  Armin Bruderlin,et al.  Perceiving affect from arm movement , 2001, Cognition.

[20]  Randolph R. Cornelius,et al.  The science of emotion: Research and tradition in the psychology of emotion. , 1997 .

[21]  Antonio Camurri,et al.  Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques , 2003, Int. J. Hum. Comput. Stud..