Human Face Analysis: From Identity to Emotion and Intention Recognition

Face recognition is the most natural mean of recognition by humans. At the same time, images (and videos) of human faces can be captured without the user's awareness. The entertainment media and science fiction has greatly contributed in shaping the public view of these technologies, most of the times exaggerating the potential impact in one's privacy. Even though face images can be acquired, in any place, with hidden cameras it is also true that face recognition technology is not dangerous per se. Rather, whenever properly deployed, it can result for the protection of the citizens and also enhance the user convenience. Face recognition today has achieved a quite high performance rate and most of the problems hindering the use of this technology have now been solved. Faces can be analyzed and characterized on the basis of several features. Then, a face can be tagged with several properties, not only the bearer's identity, but also his gender, approximate age and possible familiarity with others. Moreover, the analysis of the facial expression may also lead to understanding the mood, maybe the emotional state and intentions of the analyzed subject. May this lead to a ”Big Brother scenario”? Is this technology going to hinder a person's freedom or privacy? These questions are still to be answered and mostly depend on tomorrow's good use of this emerging technology. As for today, many scenarios can be envisaged where face recognition technologies can be fruitfully applied. Among them, the border control at airports and other ports of entry are just the most addressed in the recent past. Other applications still exist which have been overlooked and are yet worth a more extensive study and deployment from both the Academia and Industry.

[1]  Russell Beale,et al.  Affect and Emotion in Human-Computer Interaction, From Theory to Applications , 2008, Affect and Emotion in Human-Computer Interaction.

[2]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[3]  J. Movshon,et al.  Neural Foundations of Visual Motion Perception , 1992 .

[4]  Massimo Tistarelli,et al.  Multiple Constraints to Compute Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Benoit Huet,et al.  Evidence Theory-Based Multimodal Emotion Recognition , 2009, MMM.

[6]  Enrico Grosso,et al.  Dynamic face recognition: From human to machine vision , 2009, Image Vis. Comput..

[7]  Hiroshi Murase,et al.  VQ-faces - unsupervised face recognition from image sequences , 2002, Proceedings. International Conference on Image Processing.

[8]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[9]  Nicu Sebe,et al.  Emotion Recognition Based on Joint Visual and Audio Cues , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[10]  P. Ekman,et al.  Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. , 1994, Psychological bulletin.

[11]  Hatice Gunes,et al.  A Bimodal Face and Body Gesture Database for Automatic Analysis of Human Nonverbal Affective Behavior , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Matthieu Guillaumin,et al.  Combining Image-Level and Segment-Level Models for Automatic Annotation , 2012, MMM.

[13]  A. Calder.  Facial Emotion Recognition after Bilateral Amygdala Damage: Differentially Severe Impairment of Fear , 1996 .

[14]  Nicu Sebe,et al.  Learning probabilistic classifiers for human–computer interaction applications , 2005, Multimedia Systems.

[15]  Simon M. Lucas,et al.  Continuous n-tuple classifier and its application to real-time face recognition , 1998 .

[16]  Rosalind W. Picard.  Affective Computing , 1997 .

[17]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[18]  Roberto Cipolla,et al.  Face Recognition from Face Motion Manifolds using Robust Kernel Resistor-Average Distance , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[19]  Hiroshi Murase,et al.  Unsupervised recognition of multi-view face sequences based on pairwise clustering with attraction and repulsion , 2003, Comput. Vis. Image Underst..

[20]  Hatice Gunes,et al.  From the Lab to the real world: affect recognition using multiple cues and modalities , 2008 .

[21]  Hilary Buxton,et al.  Towards unconstrained face recognition from image sequences , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[22]  Gunnar Rätsch,et al.  Invariant Feature Extraction and Classification in Kernel Spaces , 1999, NIPS.

[23]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.

[24]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[26]  Shaogang Gong,et al.  Support vector regression and classification based multi-view face detection and recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[27]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[28]  Loïc Kessous,et al.  Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech , 2008, Affect and Emotion in Human-Computer Interaction.

[29]  Lawrence S. Chen,et al.  Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction , 2000 .

[30]  A. Damasio,et al.  Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity , 1988, Neurology.

[31]  Joydeep Ghosh,et al.  Multiclassifier Systems: Back to the Future , 2002, Multiple Classifier Systems.

[32]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[33]  Shaogang Gong,et al.  Modelling faces dynamically across views and over time , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Thomas S. Huang,et al.  Emotional expressions in audiovisual human computer interaction , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[35]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[36]  Caifeng Shan,et al.  Recognizing Facial Expressions Automatically from Video , 2010, Handbook of Ambient Intelligence and Smart Environments.

[37]  A. Young,et al.  Understanding face recognition. , 1986, British journal of psychology.

[38]  Léon J. M. Rothkrantz,et al.  Automatic bi-modal emotion recognition system based on fusion of facial expressions and emotion extraction from speech , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[39]  Juan Carlos Augusto,et al.  Handbook of Ambient Intelligence and Smart Environments , 2009 .

[40]  Glyn W. Humphreys,et al.  Expression is computed separately from facial identity, and it is computed separately for moving and static faces: Neuropsychological evidence , 1993, Neuropsychologia.

[41]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[42]  Stan Z. Li,et al.  Advances in Biometrics, International Conference, ICB 2007, Seoul, Korea, August 27-29, 2007, Proceedings , 2007, ICB.

[43]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Enrico Grosso,et al.  Person Authentication from Video of Faces: A Behavioral and Physiological Approach Using Pseudo Hierarchical Hidden Markov Models , 2006, ICB.

[45]  Hiroshi Murase,et al.  Unsupervised face recognition from image sequences , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[46]  Hatice Gunes,et al.  Automatic Temporal Segment Detection and Affect Recognition From Face and Body Display , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[47]  Eric Patterson,et al.  Emotion recognition using facial expressions with active appearance models , 2008 .

[48]  T. Allison,et al.  Social perception from visual cues: role of the STS region , 2000, Trends in Cognitive Sciences.

[49]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[50]  E. Rolls,et al.  Face and voice expression identification in patients with emotional and behavioural changes following ventral frontal lobe damage , 1996, Neuropsychologia.

[51]  Simon M. Lucas,et al.  Sequence recognition with scanning N-tuple ensembles , 2004, ICPR 2004.