Comparing a novel model based on the transferable belief model with humans during the recognition of partially occluded facial expressions.

Humans recognize basic facial expressions effortlessly. Yet, despite a considerable amount of research, this task remains elusive for computer vision systems. Here, we compared the behavior of one of the best computer models of facial expression recognition (Z. Hammal, L. Couvreur, A. Caplier, & M. Rombaut, 2007) with the behavior of human observers during the M. Smith, G. Cottrell, F. Gosselin, and P. G. Schyns (2005) facial expression recognition task performed on stimuli randomly sampled using Gaussian apertures. The model--which we had to significantly modify in order to give the ability to deal with partially occluded stimuli--classifies the six basic facial expressions (Happiness, Fear, Sadness, Surprise, Anger, and Disgust) plus Neutral from static images based on the permanent facial feature deformations and the Transferable Belief Model (TBM). Three simulations demonstrated the suitability of the TBM-based model to deal with partially occluded facial parts and revealed the differences between the facial information used by humans and by the model. This opens promising perspectives for the future development of the model.

[1]  F. Prêteux,et al.  MPEG-4 compliant tracking of facial features in video sequences , 2022 .

[2]  Maja Pantic,et al.  Face for Ambient Interface , 2006, Ambient Intelligence in Everyday.

[3]  Franck Davoine,et al.  Facial expression recognition and synthesis based on an appearance model , 2004, Signal Process. Image Commun..

[4]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  T. Denœux,et al.  Refined sensor tuning in the belief function framework using contextual discounting , 2006 .

[6]  Gwen Littlewort,et al.  Fully Automatic Facial Action Recognition in Spontaneous Behavior , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[7]  Maja Pantic,et al.  Detecting facial actions and their temporal segments in nearly frontal-view face image sequences , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[8]  M. Rombaut,et al.  A fusion process based on belief theory for classification of facial basic emotions , 2005, 2005 7th International Conference on Information Fusion.

[9]  P. Smets.  Data fusion in the transferable belief model , 2000, Proceedings of the Third International Conference on Information Fusion.

[10]  Kostas Karpouzis,et al.  A fuzzy system for emotion classification based on the MPEG-4 facial definition parameter set , 2000, 2000 10th European Signal Processing Conference.

[11]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[12]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[14]  Laurent Bonnaud,et al.  Belief Theory-Based Classifiers Comparison for Static Human Body Postures Recognition in Video , 2005, WEC.

[15]  Franck Davoine,et al.  Appearance factorization for facial expression analysis , 2004, BMVC.

[16]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  K Gouta,et al.  [Emotion recognition: facial components associated with various emotions]. , 2000, Shinrigaku kenkyu : The Japanese journal of psychology.

[18]  Montse Pardàs,et al.  Facial animation parameters extraction and expression recognition using Hidden Markov Models , 2002, Signal Process. Image Commun..

[19]  Zakia Hammal,et al.  Parametric models for facial features segmentation , 2006, Signal Process..

[20]  Jeffrey F. Cohn,et al.  Foundations of human computing: facial expression and emotion , 2006, ICMI '06.

[21]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[22]  J. N. Bassili.  Facial motion in the perception of faces and of emotional expression. , 1978, Journal of experimental psychology. Human perception and performance.

[23]  C. Izard.  Innate and universal facial expressions: evidence from developmental and cross-cultural research. , 1994, Psychological bulletin.

[24]  A. Murat Tekalp,et al.  Face and 2-D mesh animation in MPEG-4 , 2000, Signal Process. Image Commun..

[25]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[26]  Nicu Sebe,et al.  Learning Bayesian network classifiers for facial expression recognition both labeled and unlabeled data , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  M. Bartlett,et al.  Machine Analysis of Facial Expressions , 2007 .

[29]  P. Ekman,et al.  Facial areas and emotional information. , 1975, The Journal of communication.

[30]  JiQiang,et al.  Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequences , 2005 .

[31]  Denis Pellerin,et al.  State Filtering and Change Detection Using TBM Conflict Application to Human Action Recognition in Athletics Videos , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Garrison W. Cottrell,et al.  Transmitting and Decoding Facial Expressions , 2005, Psychological science.

[33]  Qiang Ji,et al.  Active and dynamic information fusion for facial expression understanding from image sequences , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[35]  T. Denœux.  Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence , 2008 .

[36]  Ian R. Fasel,et al.  A generative framework for real time object detection and classification , 2005, Comput. Vis. Image Underst..

[37]  Dov M. Gabbay,et al.  Handbook of defeasible reasoning and uncertainty management systems: volume 2: reasoning with actual and potential contradictions , 1998 .

[38]  Maja Pantic,et al.  Expert system for automatic analysis of facial expressions , 2000, Image Vis. Comput..

[39]  P. Zimmermann.  Automatic analysis , 2000 .

[40]  Zakia Hammal,et al.  Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model , 2007, Int. J. Approx. Reason..

[41]  D. Perrett,et al.  Facial expression megamix: Tests of dimensional and category accounts of emotion recognition , 1997, Cognition.

[42]  Jeffrey F. Cohn,et al.  The Timing of Facial Motion in posed and Spontaneous Smiles , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[43]  Jeffrey F. Cohn,et al.  Foundations of human computing: facial expression and emotion , 2006, ICMI '06.

[44]  Jeffrey F. Cohn,et al.  The Timing of Facial Motion in posed and Spontaneous Smiles , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[45]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[46]  Pascal Belin,et al.  A dynamic facial expression database , 2010 .

[47]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Lionel Valet,et al.  A statistical overview of recent literature in information fusion , 2001 .

[49]  Qiang Ji,et al.  Inferring Facial Action Units with Causal Relations , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[50]  Garrison W. Cottrell,et al.  Transmitting and decoding facial expressions of emotion , 2004 .

[51]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[53]  Philippe Smets,et al.  The Transferable Belief Model , 1991, Artif. Intell..

[54]  Takeo Kanade,et al.  Subtly different facial expression recognition and expression intensity estimation , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[55]  Philippe Smets,et al.  Decision making in the TBM: the necessity of the pignistic transformation , 2005, Int. J. Approx. Reason..

[56]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[57]  Philippe Smets,et al.  The Transferable Belief Model for Quantified Belief Representation , 1998 .

[58]  Thierry Denoeux.  Pattern Recognition and Information Fusion Using Belief Functions: Some Recent Developments , 2007, ECSQARU.

[59]  P. Smets,et al.  Assessing sensor reliability for multisensor data fusion within the transferable belief model , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[60]  Bruce Bowe.  The Face of Emotion , 1985 .

[61]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[62]  R. Adolphs,et al.  Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala , 1994, Nature.

[63]  J. Cohn,et al.  Movement Differences between Deliberate and Spontaneous Facial Expressions: Zygomaticus Major Action in Smiling , 2006, Journal of nonverbal behavior.

[64]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[65]  Thierry Denoeux,et al.  Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence , 2008, Artif. Intell..