Annealed Label Transfer for Face Expression Recognition

In this paper we propose a method for recognizing facial expressions using information from a pair of domains: one has labelled data and one with unlabelled data. As the two domains may differ in distribution, we depart from the traditional semi–supervised framework towards a transfer learning approach. In our method, which we call Annealed Label Transfer, the deep learner explores and predicts labels on the unsupervised part, yet, in order to prevent too much confidence in its predictions (as domains are not identical), the global error is regularized with a randomization input via an annealing process. The method’s evaluation is carried out on a set of four scenarios. The first two are standard benchmarks with expression faces in the wild, while the latter two have been little attempted before: face expression recognition in children and the study of the separability of anxiety-originated expressions in the wild. In all cases we show the superiority of the proposed method with respect to the strong baselines.

[1]  Alex Krizhevsky,et al.  One weird trick for parallelizing convolutional neural networks , 2014, ArXiv.

[2]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[3]  T. Sejnowski,et al.  Measuring facial expressions by computer image analysis. , 1999, Psychophysiology.

[4]  Cha Zhang,et al.  Image based Static Facial Expression Recognition with Multiple Deep Network Learning , 2015, ICMI.

[5]  Saïda Bouakaz,et al.  A novel database of Children's Spontaneous Facial Expressions (LIRIS-CSE) , 2018, Image Vis. Comput..

[6]  Davide Carneiro,et al.  Multimodal behavioral analysis for non-invasive stress detection , 2012, Expert Syst. Appl..

[7]  Zhigang Zhu,et al.  Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Patrick Shafto,et al.  Towards Automated Classification of Emotional Facial Expressions , 2017, CogSci.

[9]  J. Movellan,et al.  Human and computer recognition of facial expressions of emotion , 2007, Neuropsychologia.

[10]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[12]  Manolis Tsiknakis,et al.  Stress and anxiety detection using facial cues from videos , 2017, Biomed. Signal Process. Control..

[13]  Somnath Paul,et al.  Event-driven random backpropagation: Enabling neuromorphic deep learning machines , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[14]  Fabien Ringeval,et al.  Leveraging Unlabeled Data for Emotion Recognition With Enhanced Collaborative Semi-Supervised Learning , 2018, IEEE Access.

[15]  Honghai Liu,et al.  Feature Selection Mechanism in CNNs for Facial Expression Recognition , 2018, BMVC.

[16]  Dong-Hyun Lee,et al.  Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks , 2013 .

[17]  Tien Ho-Phuoc,et al.  CIFAR10 to Compare Visual Recognition Performance between Deep Neural Networks and Humans , 2018, ArXiv.

[18]  Adrian P Burgess,et al.  A facial expression for anxiety. , 2012, Journal of personality and social psychology.

[19]  Andrea Cavallaro,et al.  Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Emad Barsoum,et al.  Training deep networks for facial expression recognition with crowd-sourced label distribution , 2016, ICMI.

[21]  Shan Li,et al.  Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition , 2019, IEEE Transactions on Image Processing.

[22]  Jacob Jackson,et al.  Semi-Supervised Learning by Label Gradient Alignment , 2019, ArXiv.

[23]  Shiguang Shan,et al.  Facial Expression Recognition with Inconsistently Annotated Datasets , 2018, ECCV.

[24]  Ira Kemelmacher-Shlizerman,et al.  The MegaFace Benchmark: 1 Million Faces for Recognition at Scale , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Luc Van Gool,et al.  Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks , 2016, International Journal of Computer Vision.

[27]  Hyojin Kim,et al.  Facial Expression Recognition Using a Large Out-of-Context Dataset , 2018, 2018 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[28]  Yann Ollivier,et al.  Learning with Random Learning Rates , 2019, ECML/PKDD.

[29]  Frédéric Jurie,et al.  An Occam's Razor View on Learning Audiovisual Emotion Recognition with Small Training Sets , 2018, ICMI.

[30]  Sergio Escalera,et al.  Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Shuicheng Yan,et al.  Peak-Piloted Deep Network for Facial Expression Recognition , 2016, ECCV.

[32]  Louis-Philippe Morency,et al.  EmoReact: a multimodal approach and dataset for recognizing emotional responses in children , 2016, ICMI.

[33]  Vanessa Lobue,et al.  The Child Affective Facial Expression (CAFE) set: validity and reliability from untrained adults , 2014, Front. Psychol..

[34]  Shang-Hong Lai,et al.  A Compact Deep Learning Model for Robust Facial Expression Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).