A Local-Global Graph Approach for Facial Expression Recognition

In this article, we present a local global graph (LGG) method for recognizing facial expressions from static images irrespective of different illumination conditions, shadows and cluttered backgrounds. First, a neural color constancy based skin detection procedure to detect skin in complex real world images is presented. Second, the LGG method for detecting faces and facial expressions with a maximum confidence from skin segmented images is presented. The LGG approach presented here emulates the human visual perception for face and expression detection. In general, humans first extract the most important facial features such as eyes, nose, mouth, etc. and then inter-relate them for face and facial expression representations. The LG Graph embeds both the local information (the shape of facial feature is stored within the local graph at each node) and the global information (the topology of the face). Facial expression recognition from the detected face images is obtained by comparing the LG Expression Graphs with the existing the LG expression models present in the LGG database. Experimental results on the AR database and real-world images suggest the robustness of the proposed approach for facial expression recognition

[1]  Garrison W. Cottrell,et al.  Representing Face Images for Emotion Classification , 1996, NIPS.

[2]  Takeo Kanade,et al.  Recognizing lower face action units for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[3]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[4]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Christine L. Lisetti,et al.  Facial Expression Recognition Using a Neural Network , 1998, FLAIRS.

[6]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[7]  Aggelos K. Katsaggelos,et al.  Automatic facial expression recognition using facial animation parameters and multistream HMMs , 2006, IEEE Transactions on Information Forensics and Security.

[8]  Narendra Ahuja,et al.  Dot Pattern Processing Using Voronoi Neighborhoods , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Nikolaos G. Bourbakis,et al.  Image chromatic adaptation using ANNs for skin color adaptation , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[10]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[11]  Roddy Cowie,et al.  Emotion Recognition and Synthesis Based on MPEG‐4 FAPs , 2002 .

[12]  N. Tsapatsoulis,et al.  Comparing Template-based , Feature-based and Supervised Classification of Facial Expressions from Static Images , 1999 .

[13]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[14]  Zhengyou Zhang,et al.  Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with A Multilayer Perceptron , 1999, Int. J. Pattern Recognit. Artif. Intell..

[15]  Wesley E. Snyder,et al.  Application of Affine-Invariant Fourier Descriptors to Recognition of 3-D Objects , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Marco Wiering,et al.  A Model Based Method for Automatic Facial Expression Recognition , 2005, ECML.

[21]  A. Martínez,et al.  The AR face databasae , 1998 .

[22]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Nicu Sebe,et al.  Authentic facial expression analysis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[24]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  M. Pantic,et al.  Facial action unit recognition using temporal templates , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[26]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[27]  William Robert Lee,et al.  MPEG-4 Facial Animation , 2004 .

[28]  Montse Pardàs,et al.  HMM recognition of expressions in unrestrained video intervals , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[29]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[30]  Nikolaos G. Bourbakis,et al.  A survey of skin-color modeling and detection methods , 2007, Pattern Recognit..

[31]  Bernd Girod,et al.  Model-based face tracking for view-independent facial expression recognition , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[32]  Franck Davoine,et al.  Facial expression recognition and synthesis based on an appearance model , 2004, Signal Process. Image Commun..

[33]  PanticM.,et al.  Dynamics of facial expression , 2006 .

[34]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Nikolaos G. Bourbakis,et al.  An intelligent assistant for navigation of visually impaired people , 2001, Proceedings 2nd Annual IEEE International Symposium on Bioinformatics and Bioengineering (BIBE 2001).

[36]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[37]  Nikolaos G. Bourbakis,et al.  Segmentation of Colour Images with Highlights and Shadows sing Fuzzy-like Reasoning , 2001, Pattern Analysis & Applications.

[38]  Nikolaos G. Bourbakis,et al.  Object recognition using local-global graphs , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[39]  Edwin R. Hancock,et al.  Graph Matching With a Dual-Step EM Algorithm , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Narendra Ahuja,et al.  Image representation using Voronoi tessellation , 1985, Comput. Vis. Graph. Image Process..