Facial expression recognition and synthesis based on an appearance model

This article addresses the issue of expressive face modelling using an active appearance model for facial expression recognition and synthesis. We consider the six universal emotional categories namely joy, anger, fear, disgust, sadness and surprise. After a description of the active appearance model (computed with 3 or only one PCA), we address the active appearance model contribution to automatic facial expression recognition. Then we propose a new method for analysis and synthesis allowing, from a single photo, to cancel the facial expression on a given face and to artificially synthesize novel expressions on this same face. In this last framework, we propose two facial expression modelling approaches.

[1]  Shuicheng Yan,et al.  Texture-Constrained Active Shape Models , 2002 .

[2]  P PentlandAlex,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997 .

[3]  G. Cottrell,et al.  Categorical Perception in Facial Emotion Classification , 1996 .

[4]  Horst Bischof,et al.  Multiple eigenspaces , 2002, Pattern Recognit..

[5]  Michael J. Black,et al.  Robust Principal Component Analysis for Computer Vision , 2001, ICCV.

[6]  Stan Z. Li,et al.  Direct appearance models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  T. Dalgleish,et al.  Handbook of cognition and emotion , 1999 .

[8]  Franck Davoine,et al.  A solution for facial expression representation and recognition , 2002, Signal Process. Image Commun..

[9]  Takeo Kanade,et al.  Feature-point tracking by optical flow discriminates subtle differences in facial expression , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Rogério Schmidt Feris,et al.  Real-time view-based face alignment using active wavelet networks , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[11]  Tony Ezzat,et al.  Trainable videorealistic speech animation , 2002, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[12]  Demetri Terzopoulos,et al.  Multilinear image analysis for facial recognition , 2002, Object recognition supported by user interaction for service robots.

[13]  S. Demleitner.  [Communication without words]. , 1997, Pflege aktuell.

[14]  A. Yuille,et al.  Two- and Three-Dimensional Patterns of the Face , 2001 .

[15]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[16]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[17]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[18]  Thomas S. Huang,et al.  ICA-based probabilistic local appearance models , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[19]  Michael J. Black,et al.  Robust principal component analysis for computer vision , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[21]  Franck Davoine,et al.  Expressive face recognition and synthesis , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[22]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  Timothy F. Cootes,et al.  Face Recognition Using Active Appearance Models , 1998, ECCV.

[24]  Tomaso A. Poggio,et al.  Face recognition: component-based versus global approaches , 2003, Comput. Vis. Image Underst..

[25]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[26]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[27]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[28]  Baback Moghaddam,et al.  Principal Manifolds and Probabilistic Subspaces for Visual Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Timothy F. Cootes,et al.  Statistical models of appearance for computer vision , 1999 .

[30]  Timothy F. Cootes,et al.  The Use of Active Shape Models for Locating Structures in Medical Images , 1993, IPMI.

[31]  Bruce A. Draper,et al.  Recognizing faces with PCA and ICA , 2003, Comput. Vis. Image Underst..

[32]  Penio S. Penev,et al.  Local feature analysis: A general statistical theory for object representation , 1996 .

[33]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[35]  Timothy F. Cootes,et al.  On representing edge structure for model matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[36]  Takeo Kanade,et al.  Automatically Recognizing Facial Expressions in the Spatio-Temporal Domain , 1999 .

[37]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[38]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[39]  A. Anderson,et al.  The Effects of Visibility on Dialogue and Performance in a Cooperative Problem Solving Task , 1994 .

[40]  Stan Z. Li,et al.  Face recognition using the nearest feature line method , 1999, IEEE Trans. Neural Networks.

[41]  Ming-Hsuan Yang,et al.  Face Recognition Using Kernel Methods , 2001, NIPS.

[42]  M. Rosenblum,et al.  Human emotion recognition from motion using a radial basis function network architecture , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[43]  A. Pentland,et al.  Beyond Euclidean Eigenspaces: Bayesian Matching for Visual Recognition , 1998 .

[44]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[45]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[49]  Christoph Bregler,et al.  Facial expression space learning , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[50]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  YangMing-Hsuan,et al.  Detecting Faces in Images , 2002 .

[52]  Javier R. Movellan,et al.  Towards Automatic Recognition of Spontaneous Facial Actions , 2003 .

[53]  Timothy F. Cootes,et al.  Comparing Variations on the Active Appearance Model Algorithm , 2002, BMVC.