Who Will Get the Grant?: A Multimodal Corpus for the Analysis of Conversational Behaviours in Group Interviews

In the last couple of years more and more multimodal corpora have been created. Recently many of these corpora have also included RGB-D sensors' data. However, there is to our knowledge no publicly available corpus, which combines accurate gaze-tracking, and high-quality audio recording for group discussions of varying dynamics. With a corpus that would fulfill these needs, it would be possible to investigate higher level constructs such as group involvement, individual engagement or rapport which all require multi-modal feature extraction. In the following paper we describe the design and recording of such a corpus and we provide some illustrative examples of how such a corpus might be exploited in the study of group dynamics.

[1]  O. John,et al.  Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issues. , 2008 .

[2]  Jean-Marc Odobez,et al.  A semi-automated system for accurate gaze coding in natural dyadic interactions , 2013, ICMI '13.

[3]  Jean Carletta,et al.  Detecting summarization hot spots in meetings using group level involvement and turn-taking features , 2013, INTERSPEECH.

[4]  Jean-Marc Odobez,et al.  Multi-party focus of attention recognition in meetings from head pose and multimodal contextual cues , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Juho Kannala,et al.  Joint Depth and Color Camera Calibration with Distortion Correction , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Giampiero Salvi,et al.  A gaze-based method for relating group involvement to individual engagement in multimodal multiparty dialogue , 2013, ICMI '13.

[7]  J. H. Frey,et al.  The group interview in social research , 1991 .

[8]  Jean-Marc Odobez,et al.  Geometric Generative Gaze Estimation (G3E) for Remote RGB-D Cameras , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Samy Bengio,et al.  Detecting group interest-level in meetings , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[10]  Daniel Gatica-Perez,et al.  Hire me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior , 2014, IEEE Transactions on Multimedia.

[11]  Gabriel Skantze,et al.  Head Pose Patterns in Multiparty Human-Robot Team-Building Interactions , 2013, ICSR.

[12]  Petra Wagner,et al.  D64: a corpus of richly recorded conversational interaction , 2013, Journal on Multimodal User Interfaces.

[13]  Jean-Marc Odobez,et al.  A Study on Visual Focus of Attention Recognition from Head Pose in a Meeting Room , 2006, MLMI.

[14]  Jean-Marc Odobez,et al.  Investigating the midline effect for visual focus of attention recognition , 2012, ICMI '12.

[15]  Catharine Oertel Gen Bierbach.  On the use of multimodal cues for the prediction of involvement in spontaneous conversation , 2011 .

[16]  Jean Carletta,et al.  The AMI meeting corpus , 2005 .

[17]  Hayley Hung,et al.  The idiap wolf corpus: exploring group behaviour in a competitive role-playing game , 2010, ACM Multimedia.