Real-Time Emotion Recognition from Natural Bodily Expressions in Child-Robot Interaction

Emotion perception and interpretation is one of the key desired capabilities of assistive robots, which could largely enhance the quality and naturalness in human-robot interaction. According to psychological studies, bodily communication has an important role in human social behaviours. However, it is very challenging to model such affective bodily expressions, especially in a naturalistic setting, considering the variety of expressive patterns, as well as the difficulty of acquiring reliable data. In this paper, we investigate the spontaneous dimensional emotion prediction problem in a child-robot interaction scenario. The paper presents emotion elicitation, data acquisition, 3D skeletal representation, feature design and machine learning algorithms. Experimental results have shown good predictive performance on the variation trends of emotional dimensions, especially the arousal dimension.

[1]  Maja Pantic,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING , 2022 .

[2]  Hichem Sahli,et al.  Towards Real-Time Continuous Emotion Recognition from Body Movements , 2013, HBU.

[3]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.

[4]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[5]  Wade Junek,et al.  Mind Reading: The Interactive Guide to Emotions , 2007 .

[6]  S. Preston,et al.  Empathy: Its ultimate and proximate bases. , 2001, The Behavioral and brain sciences.

[7]  Andrea Kleinsmith,et al.  A categorical approach to affective gesture recognition , 2003, Connect. Sci..

[8]  B. Gelder,et al.  Why bodies? Twelve reasons for including bodily expressions in affective neuroscience , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[9]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[10]  T. Dalgleish.  Basic Emotions , 2004 .

[11]  Peter F. Driessen,et al.  Gesture-Based Affective Computing on Motion Capture Data , 2005, ACII.

[12]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[13]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .

[14]  S. Lea,et al.  Perception of Emotion from Dynamic Point-Light Displays Represented in Dance , 1996, Perception.

[15]  K. Scherer,et al.  Bodily expression of emotion , 2009 .

[16]  Daniel Bernhardt,et al.  Emotion inference from human body motion , 2010 .

[17]  Luc Van Gool,et al.  Recognizing emotions expressed by body pose: A biologically inspired neural model , 2008, Neural Networks.

[18]  Brett Stevens,et al.  Emotional body language displayed by artificial agents , 2012, TIIS.

[19]  Hichem Sahli,et al.  Relevance Vector Machine Based Speech Emotion Recognition , 2011, ACII.

[20]  Jeff T. Larsen,et al.  Further evidence for mixed emotions. , 2011, Journal of personality and social psychology.

[21]  Sethuraman Panchanathan,et al.  Gesture segmentation in complex motion sequences , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[22]  Yiannis Demiris,et al.  Online spatio-temporal Gaussian process experts with application to tactile classification , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  J. Russell.  A circumplex model of affect. , 1980 .

[24]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[25]  Werner Verhelst,et al.  An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[26]  Hatice Gunes,et al.  Automatic Temporal Segment Detection and Affect Recognition From Face and Body Display , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[28]  Minetada Osano,et al.  Towards recognizing emotion with affective dimensions through body gestures , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[29]  Elizabeth A. Crane,et al.  Methodology for Assessing Bodily Expression of Emotion , 2010 .

[30]  Dongmei Jiang,et al.  Audio Visual Emotion Recognition Based on Triple-Stream Dynamic Bayesian Network Models , 2011, ACII.

[31]  Hichem Sahli,et al.  Natural emotion elicitation for emotion modeling in child-robot interactions , 2014, WOCCI.

[32]  Athanasios Katsamanis,et al.  Tracking continuous emotional trends of participants during affective dyadic interactions using body language and speech information , 2013, Image Vis. Comput..

[33]  A. Young,et al.  Emotion Perception from Dynamic and Static Body Expressions in Point-Light and Full-Light Displays , 2004, Perception.

[34]  S. Swinnen,et al.  Action and Emotion Recognition from Point Light Displays: An Investigation of Gender Differences , 2011, PloS one.

[35]  Hichem Sahli,et al.  Context-Independent Facial Action Unit Recognition Using Shape and Gabor Phase Information , 2011, ACII.