Robust facial emotion recognition using a temporal-reinforced approach

In this paper, a temporal-reinforced approach to enhancing emotion recognition from facial images is presented. Shape and texture models of facial images are computed by using active appearance model (AAM), from which facial feature points and geometrical feature values are extracted. The extracted features are used by relevance vector machine (RVM) to recognize emotional states. We propose a temporal analysis approach to recognizing likelihood of emotional categories, such that more subtle emotion, such as degree and ratio of basic emotional states can be obtained. Furthermore, a method is developed to map the recognition result to the arousal-valence plane (A-V Plane). Experimental results verify that the performance of emotion recognition is enhanced by the proposed method.

[1]  Eva Cerezo,et al.  Sensing facial emotions in a continuous 2D affective space , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[2]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[3]  N. Sharkey,et al.  Children, the Elderly, and Interactive Robots , 2011, IEEE Robotics & Automation Magazine.

[4]  J. Russell,et al.  Multidimensional scaling of emotional facial expressions: Similarity from preschoolers to adults. , 1985 .

[5]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[6]  R. Thayer.  The biopsychology of mood and arousal , 1989 .

[7]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[8]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[9]  Kai-Tai Song,et al.  Facial expression recognition based on mixture of basic expressions and intensities , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[10]  Qingshan Liu,et al.  Facial expression recognition using encoded dynamic features , 2007, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  James A. Russell,et al.  Multidimensional scaling of emotional facial expressions: Similarity from preschoolers to adults. , 1985 .

[12]  Kai-Tai Song,et al.  Robotic Emotional Expression Generation Based on Mood Transition and Personality Model , 2013, IEEE Transactions on Cybernetics.

[13]  Cynthia Whissell,et al.  THE DICTIONARY OF AFFECT IN LANGUAGE , 1989 .

[14]  Abdolhossein Sarrafzadeh,et al.  Computerized sales assistants: the application of computer technology to measure consumer interest - a conceptual framework , 2008 .

[15]  Hatice Gunes,et al.  Output-associative RVM regression for dimensional and continuous emotion prediction , 2011, Face and Gesture 2011.

[16]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[17]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.