Top-down Bottom-up Experiments on Detecting Co-speech Gesturing in Conversation

Automatic analysis of conversational videos and detection of gesturing and body movement of the partners is one of the areas where technology development has been rapid. This paper deals with the application of video techniques to human communication studies, and focuses on detecting communicative gesturing in conversational videos. The paper sets to investigate the top-down-bottom-up methodology, which aims to combine the two approaches used in interaction studies: the human annotation of the data and the automatic analysis of the data.

[1]  Elisabeth André,et al.  Culture-specific communication management for virtual agents , 2009, AAMAS.

[2]  Csr Young,et al.  How to Do Things With Words , 2009 .

[3]  Jens Allwood.  Dialog Coding - Function and Grammar: Göteborg Coding Schemsas , 2001 .

[4]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[5]  Costanza Navarretta,et al.  The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena , 2007, Lang. Resour. Evaluation.

[6]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Costanza Navarretta,et al.  The NOMCO Multimodal Nordic Resource - Goals and Characteristics , 2010, LREC.

[8]  Sotaro Kita,et al.  How representational gestures help speaking , 2000 .

[9]  Nick Campbell,et al.  Comparing measures of synchrony and alignment in dialogue speech timing with respect to turn-taking activity , 2010, INTERSPEECH.

[10]  Kristiina Jokinen,et al.  Recognition of Human Body Movements for Studying Engagement in Conversational Video Files , 2015 .

[11]  Kristiina Jokinen,et al.  Constructive Dialogue Modelling - Speech Interaction and Rational Agents , 2009, Wiley series in agent technology.

[12]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[13]  Stuart Adam Battersby.  Moving together : the organisation of non-verbal cues during multiparty conversation , 2011 .

[14]  Björn Schuller,et al.  Computational Paralinguistics , 2013 .

[15]  Kristiina Jokinen,et al.  Investigating Engagement - intercultural and technological aspects of the collection, analysis, and use of the Estonian Multiparty Conversational video data , 2012, LREC.