Dynamic soft encoded patterns for facial event analysis

In this paper, we propose a new feature: dynamic soft encoded pattern (DSEP) for facial event analysis. We first develop similarity features to describe complicated variations of facial appearance, which take similarities between a haar-like feature in a given image and the corresponding ones in reference images as feature vector. The reference images are selected from the apex images of facial expressions, and the k-means clustering is applied to the references. We further perform a temporal clustering on the similarity features to produce several temporal patterns along the temporal domain, and then we map the similarity features into DSEP to describe the dynamics of facial expressions, as well as to handle the issue of time resolution. Finally, boosting-based classifier is designed based on DSEPs. Different from previous works, the proposed method makes no assumption on the time resolution. The effectiveness is demonstrated by extensive experiments on the Cohn-Kanade database.

[1]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[2]  J. Cohn,et al.  Deciphering the Enigmatic Face , 2005, Psychological science.

[3]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  Takeo Kanade,et al.  Detection, tracking, and classification of action units in facial expression , 2000, Robotics Auton. Syst..

[5]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[6]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Qiang Ji,et al.  Facial event classification with task oriented dynamic Bayesian network , 2004, CVPR 2004.

[8]  M. Melamed.  Detection , 2021, SETI: Astronomy as a Contact Sport.

[9]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[10]  Matti Pietikäinen,et al.  Learning Personal Specific Facial Dynamics for Face Recognition from Videos , 2007, AMFG.

[11]  Qingshan Liu,et al.  Facial Expression Recognition using Encoded Dynamic Features , 2007, ICME.

[12]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Mohammed Yeasin,et al.  From facial expression to level of interest: a spatio-temporal approach , 2004, CVPR 2004.

[14]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, CVPR Workshops.

[15]  S. Gong,et al.  Conditional Mutual Information Based Boosting for Facial Expression Recognition , 2005 .

[16]  Alex Pentland,et al.  Facial expression recognition using a dynamic model and motion energy , 1995, Proceedings of IEEE International Conference on Computer Vision.

[17]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[18]  Lior Wolf,et al.  Kernel Feature Selection , 2003 .

[19]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Qingshan Liu,et al.  Boosting Coded Dynamic Features for Facial Action Units and Facial Expression Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[22]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[23]  Maja Pantic,et al.  Facial action recognition for facial expression analysis from static face images , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Paul A. Viola,et al.  Boosting Image Retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[26]  John Daugman,et al.  Demodulation by Complex-Valued Wavelets for Stochastic Pattern Recognition , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[27]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[28]  Shaogang Gong,et al.  Conditional Mutual Infomation Based Boosting for Facial Expression Recognition , 2005, BMVC.

[29]  C. Izard.  The face of emotion , 1971 .

[30]  A. Tversky.  Features of Similarity , 1977 .

[31]  Gwen Littlewort,et al.  Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction. , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[32]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Qiang Yang,et al.  Feature selection in a kernel space , 2007, ICML '07.

[34]  Wen Gao,et al.  3D Haar-Like Features for Pedestrian Detection , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[35]  Thomas S. Huang,et al.  Facial Expression Recognition from Video Sequences : Temporal and Static Modelling , 2002 .

[36]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[37]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[38]  Jacob Whitehill,et al.  Haar features for FACS AU recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[39]  Bernhard Fröba,et al.  Boosting a Haar-Like Feature Set for Face Verification , 2003, AVBPA.

[40]  Ying-li Tian,et al.  Evaluation of Face Resolution for Expression Analysis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[41]  Changbo Hu,et al.  Manifold of facial expression , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[42]  Ahmed M. Elgammal,et al.  Facial Expression Analysis Using Nonlinear Decomposable Generative Models , 2005, AMFG.

[43]  John McDonald,et al.  Investigating the Dynamics of Facial Expression , 2006, ISVC.

[44]  Qingshan Liu,et al.  Similarity Features for Facial Event Analysis , 2008, ECCV.

[45]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[46]  Martial Hebert,et al.  Efficient visual event detection using volumetric features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[47]  Maja Pantic,et al.  Facial Action Unit Detection using Probabilistic Actively Learned Support Vector Machines on Tracked Facial Point Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.