Modeling Dominance in Group Conversations Using Nonverbal Activity Cues

Dominance - a behavioral expression of power - is a fundamental mechanism of social interaction, expressed and perceived in conversations through spoken words and audiovisual nonverbal cues. The automatic modeling of dominance patterns from sensor data represents a relevant problem in social computing. In this paper, we present a systematic study on dominance modeling in group meetings from fully automatic nonverbal activity cues, in a multi-camera, multi-microphone setting. We investigate efficient audio and visual activity cues for the characterization of dominant behavior, analyzing single and joint modalities. Unsupervised and supervised approaches for dominance modeling are also investigated. Activity cues and models are objectively evaluated on a set of dominance-related classification tasks, derived from an analysis of the variability of human judgment of perceived dominance in group discussions. Our investigation highlights the power of relatively simple yet efficient approaches and the challenges of audiovisual integration. This constitutes the most detailed study on automatic dominance modeling in meetings to date.

[1]  Anton Nijholt,et al.  Addressee Identification in Face-to-Face Meetings , 2006, EACL.

[2]  Samy Bengio,et al.  Automatic analysis of multimodal group actions in meetings , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  J. Dillard,et al.  The sounds of dominance: Vocal precursors of perceived dominance during interpersonal influence , 2000 .

[4]  Norah E. Dunbar,et al.  NONVERBAL EXPRESSIONS OF DOMINANCE AND POWER IN HUMAN RELATIONSHIPS , 2006 .

[5]  Sumit Basu,et al.  Modeling Conversational Dynamics as a Mixed-Memory Markov Process , 2004, NIPS.

[6]  J. Dovidio,et al.  Decoding visual dominance: Attributions of power based on relative percentages of looking while speaking and looking while listening. , 1982 .

[7]  Alex Pentland,et al.  Towards Measuring Human Interactions in Conversational Settings , 2001 .

[8]  Shih-Fu Chang,et al.  Survey of compressed-domain features used in audio-visual indexing and analysis , 2003, J. Vis. Commun. Image Represent..

[9]  Norah E. Dunbar,et al.  Perceptions of power and interactional dominance in interpersonal relationships , 2005 .

[10]  Hiroshi Murase,et al.  Quantifying interpersonal influence in face-to-face conversations based on visual attention patterns , 2006, CHI Extended Abstracts.

[11]  Miguel Tavares Coimbra,et al.  Approximating optical flow within the MPEG-2 compressed domain , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Elizabeth Shriberg,et al.  Spotting "hot spots" in meetings: human judgments and prosodic cues , 2003, INTERSPEECH.

[13]  Daniel Gatica-Perez,et al.  Detection and application of influence rankings in small group meetings , 2006, ICMI '06.

[14]  Jean Carletta,et al.  The AMI Meeting Corpus: A Pre-announcement , 2005, MLMI.

[15]  Samy Bengio,et al.  Detecting group interest-level in meetings , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[16]  Alex Pentland,et al.  Socially aware, computation and communication , 2005, Computer.

[17]  V. Manusov.  The sourcebook of nonverbal measures : going beyond words , 2004 .

[18]  Mary P. Harper,et al.  A Multimodal Analysis of Floor Control in Meetings , 2006, MLMI.

[19]  Fabio Pianesi,et al.  Automatic detection of group functional roles in face to face interactions , 2006, ICMI '06.

[20]  Daniel Gatica-Perez,et al.  Analyzing Group Interactions in Conversations: a Review , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[21]  Jacques M. B. Terken,et al.  Real-Time Feedback on Nonverbal Behaviour to Enhance Social Dynamics in Small Group Meetings , 2005, MLMI.

[22]  Samy Bengio,et al.  Modeling individual and group actions in meetings with layered HMMs , 2006, IEEE Transactions on Multimedia.

[23]  Marianne Schmid Mast,et al.  Dominance as expressed and inferred through speaking time: A meta-analysis , 2002 .

[24]  Frank J. Bernieri,et al.  Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream , 2000 .

[25]  Allan Mazur,et al.  Incipient Status in Small Groups , 1979 .

[26]  Walter Bender,et al.  Influencing group participation with a shared display , 2004, CSCW.

[27]  L. Smith-Lovin,et al.  INTERRUPTIONS IN GROUP DISCUSSIONS: THE EFFECTS OF GENDER AND GROUP COMPOSITION* , 1989 .

[28]  Shaogang Gong,et al.  Modelling facial colour and identity with Gaussian mixtures , 1998, Pattern Recognit..

[29]  Chuohao Yeo,et al.  Compressed domain video processing of meetings for activity estimation in dominance classification and slide transition detection , 2008 .

[30]  Jean-Marc Odobez,et al.  Using audio and video features to classify the most dominant person in a group meeting , 2007, ACM Multimedia.

[31]  Dirk Heylen,et al.  Dominance Detection in Meetings Using Easily Obtainable Features , 2005, MLMI.

[32]  Gerald Friedland,et al.  Estimating the dominant person in multi-party conversations using speaker diarization strategies , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.