Facial Action Point Based Emotion Recognition by Principal Component Analysis

This paper proposes an alternative approach to emotion recognition of a subject from selected 36 facial action points marked at specific locations on their faces. Facial expressions obtained by the subjects enacting them are recorded, and the corresponding changes in marked action points are measured. The measurements reveal that the action points have wider variations in facial expressions containing diverse emotions. Considering 10 instances for each facial expression, and carrying the same emotion, experimented over 10 subjects, we obtain a set of 100 distance matrices, representing the distance between any two selected action points. The 100 matrices for each individual emotion are averaged, and the first principal component, representing the most prominent features of the average distance matrix is evaluated. During the recognition phase, the first Principal component obtained from the distance matrix of an unknown facial expression is evaluated, and its Euclidean distance with the first Principal component of each emotion is determined. The unknown facial expression is classified into emotion class j, if the Euclidean distance between the obtained principal component and that of j-th emotion class is minimum. Classification of 120 facial images, containing equal number of samples for six emotion classes, reveals an average classification accuracy of 92.5%, the highest being in relax and disgust and the least in fear and anger.

[1]  Mohammad T. Manzuri Shalmani,et al.  Recognizing Combinations of Facial Action Units with Different Intensity Using a Mixture of Hidden Markov Models and Neural Network , 2010, MCS.

[2]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Fumio Hara,et al.  The recognition of basic facial expressions by neural network , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[4]  Qiang Ji,et al.  Inferring Facial Action Units with Causal Relations , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Takeo Kanade,et al.  Evaluation of Gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[6]  Maja Pantic,et al.  Fully Automatic Facial Action Unit Detection and Temporal Analysis , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[7]  Fumio Hara,et al.  Measurement of the Strength of Six Basic Facial Expressions by Neural Network. , 1993 .

[8]  Marian Stewart Bartlett,et al.  Classifying Facial Action , 1995, NIPS.

[9]  Yi-Cheng Zhang,et al.  Using SVM to design facial expression recognition for shape and texture features , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[10]  A. Konar,et al.  Voice and Facial Expression Based Classification of Emotion Using Linear Support Vector Machine , 2009, 2009 Second International Conference on Developments in eSystems Engineering.

[11]  Amit Konar,et al.  Computational Intelligence: Principles, Techniques and Applications , 2005 .

[12]  Hui Zhao,et al.  Facial Complex Expression Recognition Based on Fuzzy Kernel Clustering and Support Vector Machines , 2007, Third International Conference on Natural Computation (ICNC 2007).

[13]  S. Yaacob,et al.  Japanese Face Emotions Classification Using LIP Features , 2007, Geometric Modeling and Imaging (GMAI '07).

[14]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Yimo Guo,et al.  Emotion Recognition System in Images Based On Fuzzy Neural Network and HMM , 2006, 2006 5th IEEE International Conference on Cognitive Informatics.

[16]  Okechukwu A. Uwechue.  Human Face Recognition Using Third-Order Synthetic Neural Networks , 1997 .

[17]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[18]  Maja Pantic,et al.  Combined Support Vector Machines and Hidden Markov Models for Modeling Facial Action Temporal Dynamics , 2007, ICCV-HCI.

[19]  Amit Konar,et al.  Emotion Recognition From Facial Expressions and Its Control Using Fuzzy Logic , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Chung-Lin Huang,et al.  Facial Expression Recognition Using Model-Based Feature Extraction and Action Parameters Classification , 1997, J. Vis. Commun. Image Represent..

[21]  Maja Pantic,et al.  Facial action recognition in face profile image sequences , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[22]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[23]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[24]  A. Bhavsar,et al.  Facial Expression Recognition Using Neural Classifier and Fuzzy Mapping , 2005, 2005 Annual IEEE India Conference - Indicon.

[25]  Jaechang Shim,et al.  Fuzzy Neural Networks and Fuzzy Integral Approach to Curvature-Based Component Range Facial Recognition , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[26]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[27]  Fumio Hara,et al.  Recognition of Mixed Facial Expressions by Neural Network. , 1993 .