Isolating Uncertainty of the Face Expression Recognition with the Meta-Learning Supervisor Neural Network

We investigate whether the well-known poor performance of the head-on usage of the convolutional neural networks for the facial expression recognition task may be improved in terms of reducing the false positive and false negative errors. An uncertainty isolating technique is used that introduces an additional “unknown” class. A self-attention supervisor artificial neural network is used to “learn about learning” of the underlying convolutional neural networks, in particular, to learn patterns of the underlying neural network parameters that accompany wrong or correct verdicts. A novel data set containing artistic makeup and occlusions images is used to aggravate the problem of the training data not representing the test data distribution.

[1]  Michael Goh Kah Ong,et al.  Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator , 2017, MIWAI.

[2]  Jacob Whitehill,et al.  Haar features for FACS AU recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[3]  Shan Li,et al.  Deep Facial Expression Recognition: A Survey , 2018, IEEE Transactions on Affective Computing.

[4]  Edmund Y. Lam,et al.  Facial expression recognition using deep neural networks , 2015, 2015 IEEE International Conference on Imaging Systems and Techniques (IST).

[5]  Alberto Del Bimbo,et al.  A Set of Selected SIFT Features for 3D Facial Expression Recognition , 2010, 2010 20th International Conference on Pattern Recognition.

[6]  Joaquin Vanschoren,et al.  Meta-Learning: A Survey , 2018, Automated Machine Learning.

[7]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[8]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[9]  Abdenour Hadid,et al.  Improving the recognition of faces occluded by facial accessories , 2011, Face and Gesture 2011.

[10]  Sinjini Mitra,et al.  Facial asymmetry versus facial makeup , 2018, 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC).

[11]  N. Canteras,et al.  The many paths to fear , 2012, Nature Reviews Neuroscience.

[12]  B. Prabhakaran,et al.  Facilitating fashion camouflage art , 2013, ACM Multimedia.

[13]  Christoph Busch,et al.  A Generalizable Deepfake Detector based on Neural Conditional Distribution Modelling , 2020, 2020 International Conference of the Biometrics Special Interest Group (BIOSIG).

[14]  Bing-Fei Wu,et al.  Adaptive Feature Mapping for Customizing Deep Learning Based Facial Expression Recognition Model , 2018, IEEE Access.

[15]  Shervin Minaee,et al.  Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network , 2019, Sensors.

[16]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[17]  Shiguang Shan,et al.  AU-inspired Deep Networks for Facial Expression Feature Learning , 2015, Neurocomputing.

[18]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[19]  Widodo Budiharto,et al.  Discovery and innovation of computer science technology in artificial intelligence era: The 2nd International Conference on Computer Science and Computational Intelligence, ICCSCI 2017, 13-14 October 2017, Bali, Indonesia , 2017, International Conference on Computer Science and Computational Intelligence.

[20]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[21]  C A Nelson,et al.  Learning to Learn , 2017, Encyclopedia of Machine Learning and Data Mining.

[22]  Mohammad H. Mahoor,et al.  Going deeper in facial expression recognition using deep neural networks , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Yun Fu,et al.  Face Behind Makeup , 2016, AAAI.

[24]  Weidong Shi,et al.  One-Shot GAN Generated Fake Face Detection , 2020, ArXiv.

[25]  William Briggs,et al.  Uncertainty: The Soul of Modeling, Probability & Statistics , 2016 .

[26]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[27]  Arun Ross,et al.  Automatic facial makeup detection with application in face recognition , 2013, 2013 International Conference on Biometrics (ICB).

[28]  Xianghan Zheng,et al.  Multi-disciplinary Trends in Artificial Intelligence , 2015, Lecture Notes in Computer Science.

[29]  Dewi Yanti Liliana,et al.  Enhancing CNN with Preprocessing Stage in Automatic Emotion Recognition , 2017, ICCSCI.

[30]  Janis Keuper,et al.  Scalable Hyperparameter Optimization with Lazy Gaussian Processes , 2019, 2019 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC).

[31]  Jean-Luc Dugelay,et al.  Facial cosmetics database and impact analysis on automatic face recognition , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[32]  Rama Chellappa,et al.  Disguised Faces in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[34]  J. Cacioppo,et al.  The psychophysiology of emotion. , 1993 .

[35]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[36]  Tim Kelly,et al.  Establishing Safety Criteria for Artificial Neural Networks , 2003, KES.

[37]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[38]  Sebastian Sudholt,et al.  Safety Concerns and Mitigation Approaches Regarding the Use of Deep Learning in Safety-Critical Perception Tasks , 2020, SAFECOMP Workshops.

[39]  Soo-Young Lee,et al.  Hierarchical committee of deep convolutional neural networks for robust facial expression recognition , 2016, Journal on Multimodal User Interfaces.

[40]  Arun Ross,et al.  Can facial cosmetics affect the matching accuracy of face recognition systems? , 2012, 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[41]  Georgios C. Anagnostopoulos,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2003, Lecture Notes in Computer Science.

[42]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[43]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[44]  Arun Ross,et al.  Spoofing faces using makeup: An investigative study , 2017, 2017 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA).

[45]  Alex Graves,et al.  Practical Variational Inference for Neural Networks , 2011, NIPS.

[46]  Hui Ding,et al.  Learning to Recognize Patch-Wise Consistency for Deepfake Detection , 2020, ArXiv.

[47]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[48]  Raimondo Schettini,et al.  UMB-DB: A database of partially occluded 3D faces , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[49]  Federico Tombari,et al.  Sampling-Free Epistemic Uncertainty Estimation Using Approximated Variance Propagation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[50]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[51]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[52]  Stefan Winkler,et al.  Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning , 2015, ICMI.

[53]  Stan Matwin,et al.  Interpretable Deep Convolutional Neural Networks via Meta-learning , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[54]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[55]  C. V. Jawahar,et al.  Indian Movie Face Database: A benchmark for face recognition under wide variations , 2013, 2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG).

[56]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .