Analyzing perceived empathy/antipathy based on reaction time in behavioral coordination

This study analyzes emotions established between people while interacting in face-to-face conversation. By focusing on empathy and antipathy, especially the process by which they are perceived by external observers, this paper aims to elucidate the tendency of their perception and from it develop a computational model that realizes the automatic estimation of perceived empathy/antipathy. This paper makes two main contributions. First, an experiment demonstrates that an observer's perception of an interacting pair is affected by the time lags found in their actions and reactions in facial expressions and by whether their expressions are congruent or not. For example, a congruent but delayed reaction is unlikely to be perceived as empathy. Based on our findings, we propose a probabilistic model that relates the perceived empathy/antipathy of external observers to the actions and reactions of conversation participants. An experiment is conducted on ten conversations performed by 16 women in which the perceptions of nine external observers are gathered. The results demonstrate that timing cues are useful in improving the estimation performance, especially for perceived antipathy.

[1]  Ezra Stotland,et al.  Exploratory Investigations of Empathy1 , 1969 .

[2]  Barry A. Freeman.  The social neuroscience of empathy in the theatre of global ethics , 2012 .

[3]  Kristinn R. Thórisson,et al.  Fluid Semantic Back-Channel Feedback in Dialogue: Challenges and Progress , 2007, IVA.

[4]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[5]  C. Batson.  These things called empathy: Eight related but distinct phenomena. , 2009 .

[6]  Nicole Chovil.  Discourse‐oriented facial displays in conversation , 1991 .

[7]  Daniel Gatica-Perez,et al.  Analyzing Group Interactions in Conversations: a Review , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[8]  Samuel D. Bradley,et al.  Presidential Expressions and Viewer Emotion: Counterempathic Responses to Televised Leader Displays , 2004 .

[9]  Pedro M. Domingos,et al.  Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier , 1996, ICML.

[10]  B. Englis,et al.  Expectations of Cooperation and Competition and Their Effects on Observers' Vicarious Emotional Responses , 1989 .

[11]  T. Chartrand,et al.  The chameleon effect: the perception-behavior link and social interaction. , 1999, Journal of personality and social psychology.

[12]  Sung-Hyuk Cha,et al.  On measuring the distance between histograms , 2002, Pattern Recognit..

[13]  H. Critchley,et al.  Controlling emotional expression: behavioral and neural correlates of nonimitative emotional responses. , 2008, Cerebral cortex.

[14]  D. McIntosh,et al.  More than mere mimicry? The influence of emotion on rapid facial reactions to faces. , 2007, Emotion.

[15]  U. Dimberg,et al.  Facial reactions to emotional stimuli: Automatically controlled emotional responses , 2002 .

[16]  U. Hess,et al.  The impact of social context on mimicry , 2008, Biological Psychology.

[17]  Junji Yamato,et al.  Analyzing empathetic interactions based on the probabilistic modeling of the co-occurrence patterns of facial expressions in group meetings , 2011, Face and Gesture 2011.

[18]  William Ickes,et al.  Naturalistic Social Cognition: Empathic Accuracy in Mixed-Sex Dyads , 1990 .

[19]  J. Cappella.  Mutual influence in expressive behavior: adult--adult and infant--adult dyadic interaction. , 1981, Psychological bulletin.

[20]  R. Levenson,et al.  Empathy: a physiological substrate. , 1992, Journal of personality and social psychology.

[21]  W. James Potter,et al.  Rethinking validity and reliability in content analysis , 1999 .

[22]  M. Thunberg,et al.  Rapid Facial Reactions to Emotionally Relevant Stimuli , 2007 .

[23]  S. Preston,et al.  Empathy: Its ultimate and proximate bases. , 2001, The Behavioral and brain sciences.

[24]  Kazuhiro Otsuka.  Conversation Scene Analysis [Social Sciences] , 2011, IEEE Signal Processing Magazine.

[25]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Hatice Gunes,et al.  Output-associative RVM regression for dimensional and continuous emotion prediction , 2011, Face and Gesture 2011.

[27]  Carlos Busso,et al.  IEMOCAP: interactive emotional dyadic motion capture database , 2008, Lang. Resour. Evaluation.

[28]  Louis-Philippe Morency,et al.  A probabilistic multimodal approach for predicting listener backchannels , 2009, Autonomous Agents and Multi-Agent Systems.

[29]  Junji Yamato,et al.  Understanding communicative emotions from collective external observations , 2012, CHI EA '12.

[30]  J. Burgoon,et al.  Interpersonal Adaptation: Dyadic Interaction Patterns , 1995 .

[31]  Kazuhiro Otsuka.  Multimodal Conversation Scene Analysis for Understanding People's Communicative Behaviors in Face-to-Face Meetings , 2011, HCI.