Invariant representation of facial expressions for blended expression recognition on unknown subjects

Facial expressions analysis plays an important part in emotion detection. However, having an automatic and non-intrusive system to detect blended facial expression is still a challenging problem, especially when the subject is unknown to the system. Here, we propose a method that adapts to the morphology of the subject and that is based on a new invariant representation of facial expressions. In our system, one expression is defined by its relative position to 8 other expressions. As the mode of representation is relative, we show that the resulting expression space is person-independent. The 8 expressions are synthesized for each unknown subject from plausible distortions. Recognition tasks are performed in this space with a basic algorithm. The experiments have been performed on 22 different blended expressions and on either known or unknown subjects. The recognition results on known subjects demonstrate that the representation is robust to the type of data (shape and/or texture information) and to the dimensionality of the expression space. The recognition results on 22 expressions of unknown subjects show that a dimensionality of the expression space of 4 is enough to outperform traditional methods based on active appearance models and accurately describe an expression.

[1]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[2]  Gwen Littlewort,et al.  Faces of pain: automated measurement of spontaneousallfacial expressions of genuine and posed pain , 2007, ICMI '07.

[3]  Maja Pantic,et al.  The first facial expression recognition and analysis challenge , 2011, Face and Gesture 2011.

[4]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[5]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Kaizhong Zhang,et al.  On the Editing Distance Between Undirected Acyclic Graphs , 1996, Int. J. Found. Comput. Sci..

[7]  Paul J. W. ten Hagen,et al.  Emotion Disc and Emotion Squares: Tools to Explore the Facial Expression Space , 2003, Comput. Graph. Forum.

[8]  Hirokazu Madokoro,et al.  Generation of emotional feature space based on topological characteristics of facial expression images , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[9]  Horst-Michael Groß,et al.  A real-time facial expression recognition system based on Active Appearance Models using gray images and edge images , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[10]  Hong Liu,et al.  Comparison of methods for smile deceit detection by training AU6 and AU12 simultaneously , 2012, 2012 19th IEEE International Conference on Image Processing.

[11]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Renaud Séguier,et al.  Automatic design of a control interface for a synthetic face , 2009, IUI.

[13]  Changbo Hu,et al.  Manifold of facial expression , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[14]  Simon Lucey,et al.  Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face , 2007 .

[15]  Nicu Sebe,et al.  MULTIMODAL EMOTION RECOGNITION , 2005 .

[16]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[17]  Björn W. Schuller,et al.  AVEC 2011-The First International Audio/Visual Emotion Challenge , 2011, ACII.

[18]  J. Russell.  A circumplex model of affect. , 1980 .

[19]  Sungsoo Park,et al.  Subtle facial expression recognition using motion magnification , 2009, Pattern Recognit. Lett..

[20]  B. Abboud,et al.  Bilinear factorisation for facial expression analysis and synthesis , 2005 .

[21]  T. Sørensen,et al.  A method of establishing group of equal amplitude in plant sociobiology based on similarity of species content and its application to analyses of the vegetation on Danish commons , 1948 .

[22]  Maja Pantic,et al.  Expert system for automatic analysis of facial expressions , 2000, Image Vis. Comput..

[23]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[24]  M. Turk,et al.  Probabilistic expression analysis on manifolds , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[25]  Fernando De la Torre,et al.  Facial Expression Analysis , 2011, Visual Analysis of Humans.

[26]  Remco C. Veltkamp,et al.  A survey of content based 3D shape retrieval methods , 2004, Proceedings Shape Modeling Applications, 2004..

[27]  Björn W. Schuller,et al.  AVEC 2012: the continuous audio/visual emotion challenge , 2012, ICMI '12.

[28]  Mohamed Chetouani,et al.  Robust continuous prediction of human emotions using multiscale dynamic cues , 2012, ICMI '12.

[29]  Bernd Kleinjohann,et al.  Real-Time Facial Expression Recognition Using a Fuzzy Emotion Model , 2007, 2007 IEEE International Fuzzy Systems Conference.

[30]  Narendra Ahuja,et al.  Facial expression decomposition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  P. Ekman.  Emotion in the human face , 1982 .

[32]  Kostas Karpouzis,et al.  The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data , 2007, ACII.

[33]  Michael G. Strintzis,et al.  Bilinear elastically deformable models with application to 3D face and facial expression recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[34]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[35]  Daijin Kim,et al.  Natural facial expression recognition using differential-AAM and manifold learning , 2009, Pattern Recognit..

[36]  Shaogang Gong,et al.  Appearance Manifold of Facial Expression , 2005, ICCV-HCI.

[37]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[38]  Horst Bunke,et al.  A graph distance metric based on the maximal common subgraph , 1998, Pattern Recognit. Lett..

[39]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[42]  P. Ekman.  Universals and cultural differences in facial expressions of emotion. , 1972 .

[43]  Jeffrey F. Cohn,et al.  Dynamics of facial expression: normative characteristics and individual differences , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[44]  A. Ardeshir Goshtasby,et al.  Piecewise linear mapping functions for image registration , 1986, Pattern Recognit..

[45]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).