An analysis of multimodal cues of interruption in dyadic spoken interactions

Interruptions are integral elements of natural spontaneous human interaction. Both competitive and cooperative interruption serve a distinct role in the flow of conversation. This paper analyzes their differences with features, change and activeness, employing audio, visual, and disfluency data. These features are able to capture differences between the two types of interruptions better than average feature values of any single modality. Also, discriminant analysis shows that the use of multimodal cues provides a 21% improvement in classification accuracy between the two types of interruptions relative to the baseline while any individual single modality cue does not provide significant improvement.

[1]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[2]  D. H. Zimmerman,et al.  9. Sex roles, interruptions and silences in conversation , 1996 .

[3]  Michael Kipp,et al.  ANVIL - a generic annotation tool for multimodal dialogue , 2001, INTERSPEECH.

[4]  Lisa Slattery Rashotte,et al.  Measuring Interruption: Syntactic and Contextual Methods of Coding Conversation , 2002 .

[5]  B. Thorne,et al.  Language and Sex: Difference and Dominance , 1975 .

[6]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[7]  J. Goldberg.  Interrupting the discourse on interruptions , 1990 .

[8]  Carlos Busso,et al.  Real-Time Monitoring of Participants' Interaction in a Meeting using Audio-Visual Sensors , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[9]  Carlos Busso,et al.  IEMOCAP: interactive emotional dyadic motion capture database , 2008, Lang. Resour. Evaluation.

[10]  H. Li,et al.  Interruption and Involvement in Discourse: Can Intercultural Interlocutors be Trained? , 2005 .

[11]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[12]  D. Heylen.  Challenges ahead: head movements and other social acts during conversations , 2005 .

[13]  Fan Yang,et al.  Avoiding and Resolving Initiative Conflicts in Dialogue , 2007, NAACL.

[14]  James F. Allen,et al.  Speech repains, intonational phrases, and discourse markers: modeling speakers’ utterances in spoken dialogue , 1999, CL.

[15]  Li-chiung Yang.  Visualizing Spoken Discourse: Prosodic Form and Discourse Functions of Interruptions , 2001, SIGDIAL Workshop.

[16]  Zhigang Deng,et al.  Rigid Head Motion in Expressive Speech Animation: Analysis and Synthesis , 2007, IEEE Transactions on Audio, Speech, and Language Processing.