Feature selection of facial displays for detection of non verbal communication in natural conversation

Recognition of human communication has previously focused on deliberately acted emotions or in structured or artificial social contexts. This makes the result hard to apply to realistic social situations. This paper describes the recording of spontaneous human communication in a specific and common social situation: conversation between two people. The clips are then annotated by multiple observers to reduce individual variations in interpretation of social signals. Temporal and static features are generated from tracking using heuristic and algorithmic methods. Optimal features for classifying examples of spontaneous communication signals are then extracted by AdaBoost. The performance of the boosted classifier is comparable to human performance for some communication signals, even on this challenging and realistic data set.

[1]  Simeon Keates,et al.  Temporal context and the recognition of emotion from facial expression , 2003 .

[2]  Dirk Heylen,et al.  On the Contextual Analysis of Agreement Scores , 2009, Multimodal Corpora.

[3]  S. Reeves,et al.  Discourse Analysis , 2018, Understanding Communication Research Methods.

[4]  Jean-Claude Martin,et al.  Coding Emotional Events in Audiovisual Corpora , 2008, LREC.

[5]  Maja Pantic,et al.  Audiovisual laughter detection based on temporal features , 2008, ICMI '08.

[6]  Richard Bowden,et al.  Robust Lip-Tracking using Rigid Flocks of Selected Linear Predictors , 2008 .

[7]  Jean Carletta,et al.  Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus , 2007, Lang. Resour. Evaluation.

[8]  Zicheng Liu,et al.  Robust head motion computation by taking advantage of physical properties , 2000, Proceedings Workshop on Human Motion.

[9]  Takeo Kanade,et al.  Automated facial expression recognition based on FACS action units , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[11]  T. Dalgleish.  Basic Emotions , 2004 .

[12]  Roddy Cowie.  Building the databases needed to understand rich, spontaneous human behaviour , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[13]  Yuxiao Hu,et al.  Spontaneous Emotional Facial Expression Detection , 2006, J. Multim..

[14]  Peter Robinson,et al.  Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.