On the relationship between head pose, social attention and personality prediction for unstructured and dynamic group interactions

Correlates between social attention and personality traits have been widely acknowledged in social psychology studies. Head pose has commonly been employed as a proxy for determining the social attention direction in small group interactions. However, the impact of head pose estimation errors on personality estimates has not been studied to our knowledge. In this work, we consider the unstructured and dynamic cocktail party scenario where the scene is captured by multiple, large field-of-view cameras. Head pose estimation is a challenging task under these conditions owing to the uninhibited motion of persons (due to which facial appearance varies owing to perspective and scale changes), and the low resolution of captured faces. Based on proxemic and social attention features computed from position and head pose annotations, we first demonstrate that social attention features are excellent predictors of the Extraversion and Neuroticism personality traits. We then repeat classification experiments with behavioral features computed from automated estimates-- obtained experimental results show that while prediction performance for both traits is affected by head pose estimation errors, the impact is more adverse for Extraversion.

[1]  Subramanian Ramanathan,et al.  Putting the pieces together: multimodal analysis of social attention in meetings , 2010, ACM Multimedia.

[2]  Chuohao Yeo,et al.  Modeling Dominance in Group Conversations Using Nonverbal Activity Cues , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[3]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[4]  E. Hall,et al.  The Hidden Dimension , 1970 .

[5]  S De Julio,et al.  Neuroticism and Proxemic Behavior , 1977, Perceptual and motor skills.

[6]  Jean-Marc Odobez,et al.  Recognizing Visual Focus of Attention From Head Pose in Natural Meetings , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Nicu Sebe,et al.  The SocioMetric Badges Corpus: A Multilevel Behavioral Dataset for Social Behavior in Complex Organizations , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[8]  M. Perugini,et al.  The Big Five Marker Scales (BFMS) and the Italian AB5C taxonomy: Analyses from an emic-etic perspective , 2002 .

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Jean-Marc Odobez,et al.  A Cognitive and Unsupervised Map Adaptation Approach to the Recognition of the Focus of Attention from Head Pose , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[11]  Jean-Marc Odobez,et al.  Investigating automatic dominance estimation in groups from visual attention and speaking activity , 2008, ICMI '08.

[12]  Judith A. Hall,et al.  A thin slice perspective on the accuracy of first impressions , 2007 .

[13]  Subramanian Ramanathan,et al.  An Adaptation Framework for Head-Pose Classification in Dynamic Multi-view Scenarios , 2012, ACCV.

[14]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[15]  Rainer Stiefelhagen,et al.  Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios , 2008, ICMI '08.

[16]  Roberto Brunelli,et al.  Dynamic Head Location and Pose from Video , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[17]  Elisa Ricci,et al.  Space speaks: towards socially and personality aware visual surveillance , 2010, MPVA '10.

[18]  Alexander H. Waibel,et al.  Modeling focus of attention for meeting indexing based on multiple cues , 2002, IEEE Trans. Neural Networks.

[19]  Tapio Salakoski,et al.  A comparison of AUC estimators in small-sample studies , 2009, MLSB.

[20]  Subramanian Ramanathan,et al.  Automatic modeling of personality states in small group interactions , 2011, MM '11.

[21]  V. Bruce,et al.  Do the eyes have it? Cues to the direction of social attention , 2000, Trends in Cognitive Sciences.

[22]  Subramanian Ramanathan,et al.  Connecting Meeting Behavior with Extraversion—A Systematic Study , 2012, IEEE Transactions on Affective Computing.

[23]  Bernhard Pfahringer,et al.  Locally Weighted Naive Bayes , 2002, UAI.

[24]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[25]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[26]  J. Dovidio,et al.  Decoding visual dominance: Attributions of power based on relative percentages of looking while speaking and looking while listening. , 1982 .

[27]  Vladimir Cherkassky,et al.  Connection between SVM+ and multi-task learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).