Classification of cooperative and competitive overlaps in speech using cues from the context, overlapper, and overlappee

One of the major properties of overlapping speech is that it can be perceived as competitive or cooperative. For the development of real-time spoken dialog systems and the analysis of affective and social human behavior in conversations, it is important to (automatically) distinguish between these two types of overlap. We investigate acoustic characteristics of cooperative and competitive overlaps with the aim to develop automatic classifiers for the classification of overlaps. In addition to acoustic features, we also use information from gaze and head movement annotations. Contexts preceding and during the overlap are taken into account, as well as the behaviors of both the overlapper and the overlappee. We compare various feature sets in classification experiments that are performed on the AMI corpus. The best performances obtained lie around 27%–30% EER.

[1]  Jean Carletta,et al.  Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus , 2007, Lang. Resour. Evaluation.

[2]  Guy J. Brown,et al.  Resources for turn competition in overlap in multi-party conversations: speech rate, pausing and duration , 2010, INTERSPEECH.

[3]  K. Murata.  Intrusive or co-operative? A cross-cultural study of interruption , 1994 .

[4]  Peter French,et al.  Turn-competitive incomings , 1983 .

[5]  Andreas Stolcke,et al.  Can Prosody Aid the Automatic Processing of Multi-Party Meetings? Evidence from Predicting Punctuation, Disfluencies, and Overlapping Speech , 2003 .

[6]  Li-chiung Yang.  Visualizing Spoken Discourse: Prosodic Form and Discourse Functions of Interruptions , 2001, SIGDIAL Workshop.

[7]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[8]  S. MacFarlane,et al.  Prosody as an Interactional Resource: Turn-projection and Overlap , 1998, Language and speech.

[9]  J. Goldberg.  Interrupting the discourse on interruptions , 1990 .

[10]  Julia Hirschberg,et al.  A Corpus-Based Study of Interruptions in Spoken Dialogue , 2012, INTERSPEECH.

[11]  Catharine Oertel,et al.  Context Cues For Classification Of Competitive And Collaborative Overlaps , 2012 .

[12]  H. Li.  Cooperative and Intrusive Interruptions in Inter- and Intracultural Dyadic Discourse , 2001 .

[13]  J. Sundberg,et al.  Perceptual and acoustic correlates of abnormal voice qualities. , 1980, Acta oto-laryngologica.

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[15]  Shrikanth S. Narayanan,et al.  An analysis of multimodal cues of interruption in dyadic spoken interactions , 2008, INTERSPEECH.

[16]  Shrikanth S. Narayanan,et al.  Predicting interruptions in dyadic spoken interactions , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.