Facial animation parameters extraction and expression recognition using Hidden Markov Models

Abstract The video analysis system described in this paper aims at facial expression recognition consistent with the MPEG4 standardized parameters for facial animation, FAP. For this reason, two levels of analysis are necessary: low-level analysis to extract the MPEG4 compliant parameters and high-level analysis to estimate the expression of the sequence using these low-level parameters. The low-level analysis is based on an improved active contour algorithm that uses high level information based on principal component analysis to locate the most significant contours of the face (eyebrows and mouth), and on motion estimation to track them. The high-level analysis takes as input the FAP produced by the low-level analysis tool and, by means of a Hidden Markov Model classifier, detects the expression of the sequence.

[1]  Shiu Yin Yuen,et al.  An unbiased active contour algorithm for object tracking , 1998, Pattern Recognit. Lett..

[2]  A. Young,et al.  Handbook of Research on Face Processing , 1989 .

[3]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[4]  Peter Eisert,et al.  Analyzing Facial Expressions for Virtual Conferencing , 1998, IEEE Computer Graphics and Applications.

[5]  Alex Pentland,et al.  Facial expression recognition using a dynamic model and motion energy , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Justine Cassell,et al.  Embodied Conversation: Integrating Face and Gesture into Automatic Spoken Dialogue Systems , 1998 .

[7]  Garrison W. Cottrell,et al.  EMPATH: Face, Emotion, and Gender Recognition Using Holons , 1990, NIPS.

[8]  Steve R. Gunn,et al.  Global and Local Active Contours for Head Boundary Extraction , 2004, International Journal of Computer Vision.

[9]  Alok Gupta,et al.  Dynamic Programming for Detecting, Tracking, and Matching Deformable Contours , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[11]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..

[12]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[14]  Irfan Essa,et al.  Analysis, interpretation and synthesis of facial expressions , 1995 .

[15]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[16]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Patrick Pérez,et al.  Generalized Likelihood Ratio-based Face Detection and Extraction of Mouth Features , 1997, AVBPA.

[18]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[19]  Dimitris N. Metaxas,et al.  Adjusting Shape Parameters Using Model-Based Optical Flow Residuals , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Ramesh C. Jain,et al.  Using Dynamic Programming for Solving Variational Problems in Vision , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Jörgen Ahlberg,et al.  An Active Model for Facial Feature Tracking , 2002, EURASIP J. Adv. Signal Process..

[22]  Alex Pentland,et al.  Probabilistic visual learning for object detection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[23]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[25]  Larry S. Davis,et al.  A probabilistic framework for rigid and non-rigid appearance based tracking and recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[26]  Montse Pardàs,et al.  Motion estimation based tracking of active contours , 2001, Pattern Recognit. Lett..

[27]  Takeo Kanade,et al.  Subtly different facial expression recognition and expression intensity estimation , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[28]  Verónica Vilaplana,et al.  Region-based segmentation and tracking of human faces , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[29]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.