Sayette Group Formation Task (GFT) Spontaneous Facial Expression Database

Despite the important role that facial expressionsplay in interpersonal communication and our knowledge thatinterpersonal behavior is influenced by social context, nocurrently available facial expression database includes multipleinteracting participants. The Sayette Group Formation Task(GFT) database addresses the need for well-annotated videoof multiple participants during unscripted interactions. Thedatabase includes 172,800 video frames from 96 participantsin 32 three-person groups. To aid in the development ofautomated facial expression analysis systems, GFT includesexpert annotations of FACS occurrence and intensity, faciallandmark tracking, and baseline results for linear SVM, deeplearning, active patch learning, and personalized classification.Baseline performance is quantified and compared using identicalpartitioning and a variety of metrics (including meansand confidence intervals). The highest performance scores werefound for the deep learning and active patch learning methods.Learn more at http://osf.io/7wcyz.

[1]  Honggang Zhang,et al.  Joint Patch and Multi-label Learning for Facial Action Unit and Holistic Expression Recognition , 2016, IEEE Transactions on Image Processing.

[2]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[3]  G. Cumming,et al.  A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions , 2001 .

[4]  Takeo Kanade,et al.  Dense 3D face alignment from 2D video for real-time use , 2017, Image Vis. Comput..

[5]  Louis-Philippe Morency,et al.  A multi-label convolutional neural network approach to cross-domain action unit detection , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[6]  Qingshan Liu,et al.  Learning active facial patches for expression analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Maja Pantic,et al.  The first facial expression recognition and analysis challenge , 2011, Face and Gesture 2011.

[8]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[9]  Lijun Yin,et al.  FERA 2015 - second Facial Expression Recognition and Analysis challenge , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[10]  Shaun J. Canavan,et al.  Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Thomas R. Kirchner,et al.  Alcohol and Group Formation , 2012, Psychological science.

[12]  R. Alpert,et al.  Communications Through Limited-Response Questioning , 1954 .

[13]  A. Feinstein,et al.  High agreement but low kappa: II. Resolving the paradoxes. , 1990, Journal of clinical epidemiology.

[14]  J. Cohn,et al.  Measuring Facial Action , 2008 .

[15]  J. Rabe-Jabłońska,et al.  [Affective disorders in the fourth edition of the classification of mental disorders prepared by the American Psychiatric Association -- diagnostic and statistical manual of mental disorders]. , 1993, Psychiatria polska.

[16]  K. Gwet.  Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters , 2014 .

[17]  Daniel McDuff,et al.  Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected "In-the-Wild" , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[18]  Francisco Herrera,et al.  Learning from Imbalanced Data Sets , 2018, Springer International Publishing.

[19]  Laszlo A. Jeni,et al.  Spontaneous facial expression in unscripted social interactions can be measured automatically , 2015, Behavior research methods.

[20]  Maja Pantic,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING , 2022 .

[21]  R. Adolphs,et al.  The social brain: neural basis of social knowledge. , 2009, Annual review of psychology.

[22]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[23]  Simon Lucey,et al.  How much training data for facial action unit detection? , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[24]  Jeffrey F. Cohn,et al.  Painful data: The UNBC-McMaster shoulder pain expression archive database , 2011, Face and Gesture 2011.

[25]  Mohammad H. Mahoor,et al.  DISFA: A Spontaneous Facial Action Intensity Database , 2013, IEEE Transactions on Affective Computing.

[26]  Jean Ponce,et al.  Sparse Modeling for Image and Vision Processing , 2014, Found. Trends Comput. Graph. Vis..

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[29]  Shaun J. Canavan,et al.  BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database , 2014, Image Vis. Comput..

[30]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[31]  Fernando De la Torre,et al.  Selective Transfer Machine for Personalized Facial Expression Analysis , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  J. Cohn,et al.  Automated Face Analysis for Affective Computing , 2015 .

[33]  Shaun J. Canavan,et al.  BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database , 2014, Image Vis. Comput..

[34]  Zhang Xiong,et al.  Confidence Preserving Machine for Facial Action Unit Detection , 2015, IEEE Transactions on Image Processing.

[35]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[36]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[37]  A. J. Fridlund.  Human Facial Expression: An Evolutionary View , 1994 .

[38]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[39]  Robert C. Holte,et al.  Cost curves: An improved method for visualizing classifier performance , 2006, Machine Learning.

[40]  Dale J. Prediger,et al.  Coefficient Kappa: Some Uses, Misuses, and Alternatives , 1981 .