Recognition of Human Body Movements for Studying Engagement in Conversational Video Files

This paper investigates object recognition techniques to automatically detect human behavior in video conversations. The ViBe background subtraction algorithm, together with standard image processing techniques is applied to conversational videos where two people meet for the first time, and the results show the usefulness of the technique in human communication analysis. By detecting the conversational participants and analyzing their conversational styles through the detected body movements, we can visualize, and draw conclusions concerning the participants’ engagement in the communicative activity. The paper discusses these novel observations that show the synchrony and engagement in the participants' behavior.

[1]  C. P. Sumathi,et al.  A survey of techniques for human detection in static images , 2012, CCSEIT '12.

[2]  C. Goodwin.  Conversational Organization: Interaction Between Speakers and Hearers , 1981 .

[3]  Rama Chellappa,et al.  Audio-visual interaction in multimodal communication , 1997 .

[4]  Tobias Baur,et al.  NovA: Automated Analysis of Nonverbal Signals in Social Interactions , 2013, HBU.

[5]  Shelley Masion Rosenberg.  Bodily Communication , 1978 .

[6]  S. Kollias,et al.  Synthesizing Gesture Expressivity Based on Real Sequences , 2006 .

[7]  Costanza Navarretta,et al.  The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena , 2007, Lang. Resour. Evaluation.

[8]  I. Patras,et al.  Spatiotemporal salient points for visual recognition of human actions , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Kristiina Jokinen,et al.  Investigating Engagement - intercultural and technological aspects of the collection, analysis, and use of the Estonian Multiparty Conversational video data , 2012, LREC.

[10]  Mohamed Chetouani,et al.  Automatic Imitation Assessment in Interaction , 2012, HBU.

[11]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[12]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Kristiina Jokinen,et al.  Multimodal Signals and Holistic Interaction Structuring , 2012, COLING.

[14]  Michael Kipp,et al.  ANVIL - a generic annotation tool for multimodal dialogue , 2001, INTERSPEECH.