Action unit classification for facial expression recognition using active learning and SVM

Automatic facial expression analysis remains challenging due to its low recognition accuracy and poor robustness. In this study, we utilized active learning and support vector machine (SVM) algorithms to classify facial action units (AU) for human facial expression recognition. Active learning was used to detect the targeted facial expression AUs, while an SVM was utilized to classify different AUs and ultimately map them to their corresponding facial expressions. Active learning reduces the number of non-support vectors in the training sample set and shortens the labeling and training times without affecting the performance of the classifier, thereby reducing the cost of labeling samples and improving the training speed. Experimental results show that the proposed algorithm can effectively suppress correlated noise and achieve higher recognition rates than principal component analysis and a human observer on seven different facial expressions.

[1]  P. Ekman,et al.  Facial action coding system , 2019 .

[2]  Ielaf Osaamah Abdul-Majjed Emotion Recognition System Based on Facial Expressions Using SVM , 2017 .

[3]  Li Chen,et al.  AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detection , 2018, Neurocomputing.

[4]  Jialie Shen,et al.  On Effective Location-Aware Music Recommendation , 2016, ACM Trans. Inf. Syst..

[5]  Zhang Fang-sheng,et al.  The Face Recognition Algorithm Based on Improved LBP , 2012 .

[6]  Feng Xu,et al.  Microexpression Identification and Categorization Using a Facial Dynamics Map , 2017, IEEE Transactions on Affective Computing.

[7]  Guoying Zhao,et al.  Micro-expression Recognition Using Dynamic Textures on Tensor Independent Color Space , 2014, 2014 22nd International Conference on Pattern Recognition.

[8]  Liang Chen A Fair Comparison Should Be Based on the Same Protocol--Comments on "Trainable Convolution Filters and Their Application to Face Recognition" , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Xiaolan Fu,et al.  Face Recognition and Micro-expression Recognition Based on Discriminant Tensor Subspace Analysis Plus Extreme Learning Machine , 2014, Neural Processing Letters.

[10]  Yuan Xie,et al.  Facial Landmark Machines: A Backbone-Branches Architecture With Progressive Representation Learning , 2018, IEEE Transactions on Multimedia.

[11]  Honggang Zhang,et al.  Deep Region and Multi-label Learning for Facial Action Unit Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Matti Pietikäinen,et al.  Facial Micro-Expression Recognition Using Spatiotemporal Local Binary Pattern with Integral Projection , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[13]  Guoying Zhao,et al.  A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition , 2016, IEEE Transactions on Affective Computing.

[14]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[15]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[16]  Matti Pietikäinen,et al.  Spontaneous facial micro-expression analysis using Spatiotemporal Completed Local Quantized Patterns , 2016, Neurocomputing.

[17]  Honggang Zhang,et al.  Joint patch and multi-label learning for facial action unit detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Starr Roxanne Hiltz,et al.  Identifying Opportunities for Valuable Encounters: Toward Context-Aware Social Matching Systems , 2015, TOIS.

[19]  Yuichi Ohta,et al.  Facial Micro-Expression Detection in Hi-Speed Video Based on Facial Action Coding System (FACS) , 2013, IEICE Trans. Inf. Syst..

[20]  John See,et al.  LBP with Six Intersection Points: Reducing Redundant Information in LBP-TOP for Micro-expression Recognition , 2014, ACCV.

[21]  Moi Hoon Yap,et al.  A Review on Facial Micro-Expressions Analysis: Datasets, Features and Metrics , 2018, ArXiv.