A local experts organization model with application to face emotion recognition

This paper presents a novel approach for recognizing human facial emotion in order to further detect human suspicious behaviors. Instead of relying on relative poor representation of facial features in a flat vector form, the approach utilizes a format of tree structures with Gabor feature representations to present a facial emotional state. The novel local experts organization (LEO) model is proposed for the processing of this tree structure representation. The motivation for the LEO model is to deal with the inconsistent length of features in case there are some features failed to be detected. The proposed LEO model is inspired by the natural hierarchical model presented in natural organization, where workers (local experts) reports to their supervisor (fusion classifier), whom in turn reports to upper management (global fusion classifier). Moreover, an Asian emotion database is created. The database contains high-resolution images of 153 Asian subjects in six basic pseudo-emotions (excluding neutral expression) in three different poses for evaluating our proposed system. Empirical studies were conducted to benchmark our approach with other well-known classifiers applying to the system, and the results showed that our approach is the most robust, and less affected by noise from feature locators for the face emotion recognition system.

[1]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[2]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[3]  Chengjun Liu,et al.  Independent component analysis of Gabor features for face recognition , 2003, IEEE Trans. Neural Networks.

[4]  J. Thompson.  Development of facial expression of emotion in blind and seeing children. , 1941 .

[5]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[6]  Kar-Ann Toh,et al.  Benchmarking a reduced multivariate polynomial pattern classifier , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[8]  Xudong Jiang,et al.  A reduced multivariate polynomial model for multimodal biometrics and classifiers fusion , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Nicu Sebe,et al.  Authentic Facial Expression Analysis , 2004, FGR.

[10]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[11]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[12]  Michael B. Lewis,et al.  Capgras delusion: a window on face recognition , 2001, Trends in Cognitive Sciences.

[13]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[14]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[15]  Jia-Jun Wong,et al.  A Brain-Inspired Model for Recognizing Human Emotional States from Facial Expression , 2007 .

[16]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[17]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[19]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[20]  Marian Stewart Bartlett,et al.  Viewpoint Invariant Face Recognition using Independent Component Analysis and Attractor Networks , 1996, NIPS.

[21]  Penio S. Penev,et al.  Local feature analysis: A general statistical theory for object representation , 1996 .

[22]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[24]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[25]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  A. Damasio,et al.  Emotion, decision making and the orbitofrontal cortex. , 2000, Cerebral cortex.

[27]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[29]  A. Isen,et al.  Positive affect and decision making. , 1993 .

[30]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[31]  Andrew McCallum,et al.  A comparison of event models for naive bayes text classification , 1998, AAAI 1998.

[32]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[33]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[34]  S. Baron-Cohen.  Mindblindness: An Essay on Autism and Theory of Mind , 1997 .

[35]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[37]  Siu-Yeung Cho,et al.  Probabilistic based recursive model for adaptive processing of data structures , 2008, Expert Syst. Appl..

[38]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[39]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[40]  Ah Chung Tsoi,et al.  An improved algorithm for learning long-term dependency problems in adaptive processing of data structures , 2003, IEEE Trans. Neural Networks.

[41]  Michael David Kelly,et al.  Visual identification of people by computer , 1970 .

[42]  L. F. Barrett,et al.  Handbook of Emotions , 1993 .

[43]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[45]  T. Dalgleish,et al.  Handbook of cognition and emotion , 1999 .

[46]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[47]  Volker Blanz,et al.  Component-Based Face Recognition with 3D Morphable Models , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[48]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[50]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).