Facial Expression Recognition from Infrared Thermal Videos

In this paper, a spontaneous facial expression recognition method using infrared thermal videos is proposed. Firstly, the sequence features are extracted from the infrared thermal horizontal and vertical temperature difference sequences of different facial sub-regions. Secondly, a feature subset is selected according to their F-values. Thirdly, the Adaboost algorithm, with the weak classifiers of k-Nearest Neighbor, is used to classify facial expressions in arousal and valence dimensions. Finally, experiments on the Natural Visible and Infrared facial Expression (USTC-NVIE) database demonstrates the effectiveness of our approach.

[1]  Ioannis T. Pavlidis,et al.  StressCam: non-contact measurement of users' emotional states through thermal imaging , 2005, CHI Extended Abstracts.

[2]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Bo Wu,et al.  Real time facial expression recognition with AdaBoost , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[4]  I. Pavlidis,et al.  Thermal image analysis for polygraph testing. , 2002, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[5]  Tom Chau,et al.  Classifying Affective States Using Thermal Infrared Imaging of the Human Face , 2010, IEEE Transactions on Biomedical Engineering.

[6]  Yasunari Yoshitomi,et al.  Facial expression recognition for speaker using thermal image processing and speech recognition system , 2010 .

[7]  Qiang Ji,et al.  Multi-view face and eye detection using discriminant features , 2007, Comput. Vis. Image Underst..

[8]  Fei Chen,et al.  A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference , 2010, IEEE Transactions on Multimedia.

[9]  S. D. Jenkins,et al.  Comparing Thermographic, EEG, and Subjective Measures of Affective Experience During Simulated Product Interactions , 2009 .

[10]  Ioannis Pavlidis,et al.  The face of fear , 2001 .

[11]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[12]  Leonardo Trujillo,et al.  Automatic Feature Localization in Thermal Images for Facial Expression Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[13]  Masood Mehmood Khan,et al.  Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature , 2009, TAP.

[14]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Yasunari Yoshitomi,et al.  A facial expression recognition for a speaker of a phoneme of vowel using thermal image processing and a speech recognition system , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[16]  Akio Nozawa,et al.  Correlation analysis on alpha attenuation and nasal skin temperature , 2009 .

[17]  Rie Nakanishi,et al.  Facial skin temperature decreases in infants with joyful expression. , 2008, Infant behavior & development.

[18]  David Sander,et al.  Thermal Analysis of Facial Muscles Contractions , 2011, IEEE Transactions on Affective Computing.

[19]  M.M. Khan,et al.  Infrared Thermal Sensing of Positive and Negative Affective States , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[20]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[21]  Leonardo Trujillo,et al.  Visual learning of texture descriptors for facial expression recognition in thermal imagery , 2007, Comput. Vis. Image Underst..

[22]  Masood Mehmood Khan,et al.  Automated Facial Expression Classification and affect interpretation using infrared measurement of facial skin temperature variations , 2006, TAAS.

[23]  M.M. Khan,et al.  Automated classification and recognition of facial expressions using infrared thermal imaging , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[24]  Barbara L. O'Kane,et al.  Analysis of facial thermal variations in response to emotion: eliciting film clips , 2009, Defense + Commercial Sensing.