How Interlocutors Coordinate with each other within Emotional Segments?

In this paper, we aim to investigate the coordination of interlocutors behavior in different emotional segments. Conversational coordination between the interlocutors is the tendency of speakers to predict and adjust each other accordingly on an ongoing conversation. In order to find such a coordination, we investigated 1) lexical similarities between the speakers in each emotional segments, 2) correlation between the interlocutors using psycholinguistic features, such as linguistic styles, psychological process, personal concerns among others, and 3) relation of interlocutors turn-taking behaviors such as competitiveness. To study the degree of coordination in different emotional segments, we conducted our experiments using real dyadic conversations collected from call centers in which agent’s emotional state include empathy and customer’s emotional states include anger and frustration. Our findings suggest that the most coordination occurs between the interlocutors inside anger segments, where as, a little coordination was observed when the agent was empathic, even though an increase in the amount of non-competitive overlaps was observed. We found no significant difference between anger and frustration segment in terms of turn-taking behaviors. However, the length of pause significantly decreases in the preceding segment of anger where as it increases in the preceding segment of frustration.

[1]  Firoj Alam,et al.  Fusion of acoustic, linguistic and psycholinguistic features for Speaker Personality Traits recognition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Joann L. Robinson,et al.  Empathy and Prosocial Behavior , 2008 .

[3]  J. Russell A circumplex model of affect. , 1980 .

[4]  Ross A. Thompson,et al.  Emotion regulation: Conceptual foundations , 2007 .

[5]  Morena Danieli,et al.  Annotating and categorizing competition in overlap speech , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Maja Pantic,et al.  Social signal processing: Survey of an emerging domain , 2009, Image Vis. Comput..

[7]  K. Lambert Handbook of Emotion Regulation , 2007 .

[8]  Anna Esposito,et al.  Conflict Cues in Call Center Interactions , 2015 .

[9]  Mattias Heldner,et al.  Pauses, gaps and overlaps in conversations , 2010, J. Phonetics.

[10]  Dirk Heylen,et al.  Social Signal Processing: The Research Agenda , 2011, Visual Analysis of Humans.

[11]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[12]  Firoj Alam,et al.  Emotion Unfolding and Affective Scenes: A Case Study in Spoken Conversations , 2015, ERM4CT@ICMI.

[13]  Panayiotis G. Georgiou,et al.  Behavioral Signal Processing: Deriving Human Behavioral Informatics From Speech and Language , 2013, Proceedings of the IEEE.

[14]  Firoj Alam,et al.  Can we detect speakers' empathy?: A real-life case study , 2016, 2016 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom).

[15]  J. Gross The Emerging Field of Emotion Regulation: An Integrative Review , 1998 .

[16]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[18]  Dirk Heylen,et al.  Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing , 2012, IEEE Transactions on Affective Computing.

[19]  Firoj Alam,et al.  Unsupervised recognition and clustering of speech overlaps in spoken conversations , 2014, SLAM@INTERSPEECH.

[20]  Julia Hirschberg,et al.  Pragmatic aspects of temporal accommodation in turn-taking , 2011, Journal of Pragmatics.

[21]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[22]  G. Riccardi,et al.  AUTOMATIC SUMMARIZATION OF CALL-CENTER CONVERSATIONS , 2015 .

[23]  Morena Danieli,et al.  The role of speakers and context in classifying competition in overlapping speech , 2015, INTERSPEECH.

[24]  K. Scherer Psychological models of emotion. , 2000 .

[25]  T. Dalgleish Basic Emotions , 2004 .

[26]  Björn W. Schuller,et al.  Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge , 2011, Speech Commun..

[27]  S. A. Chowdhury,et al.  Predicting User Satisfaction from Turn-Taking in Spoken Conversations , 2016, INTERSPEECH.