Recognition of Facial Expressions in the Presence of Occlusion

We present a new approach for the recognition of facial expressions from video sequences in the presence of occlusion. Although promising results have been reported in the literature on automatic recognition of facial expressions, most techniques have been assessed using experiments performed in controlled laboratory conditions which do not reflect real-world conditions. The goal of the work presented herein, is to develop recognition techniques that will overcome some limitations of current techniques, such as their sensitivity to partial occlusion of the face. The proposed approach is based on a localised representation of facial features, and on data fusion. The experiments show that the proposed approach is robust to partial occlusion of the face.

[1]  Anil K. Jain,et al.  NOTE ON DISTANCE-WEIGHTED k-NEAREST NEIGHBOR RULES. , 1978 .

[2]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Claude C. Chibelushi,et al.  Robust Facial Feature Tracking , 2000, BMVC.

[4]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Alex Pentland,et al.  LAFTER: Lips and Face Real Time Tracker with Facial Expression Recognition , 1997, CVPR 1997.

[6]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Takeo Kanade,et al.  Detection, tracking, and classification of action units in facial expression , 2000, Robotics Auton. Syst..

[9]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[10]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  James D. Hollan,et al.  Strategic directions in human-computer interaction , 1996, CSUR.

[13]  M. Yachida,et al.  Facial expression recognition and its degree estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Ashok Samal,et al.  Automatic recognition and analysis of human faces and facial expressions: a survey , 1992, Pattern Recognit..

[15]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[16]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Farzin Deravi,et al.  A review of speech-based bimodal recognition , 2002, IEEE Trans. Multim..

[18]  J. Ohya,et al.  Recognition of facial expressions using HMM with continuous output probabilities , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[19]  S. Iyengar,et al.  Multi-Sensor Fusion: Fundamentals and Applications With Software , 1997 .

[20]  Masahiko Yachida,et al.  Expression recognition from time-sequential facial images by use of expression change model , 1997, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[21]  Jürgen Ziegler,et al.  Interactive techniques , 1996, CSUR.

[22]  Alex Pentland,et al.  LAFTER: lips and face real time tracker , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Takeo Kanade,et al.  Feature-point tracking by optical flow discriminates subtle differences in facial expression , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[24]  S. Sitharama Iyengar,et al.  Robust Distributed Comupting and Sensing Algorithm , 1996, Computer.

[25]  Manish Sarkar.  Modular pattern classifiers: a brief survey , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[26]  John S. D. Mason,et al.  Adaptive classifier integration for robust pattern recognition , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[27]  Murray R. Spiegel,et al.  Schaum's outline of theory and problems of statistics. , 1961 .

[28]  P. Ekman.  Unmasking The Face , 1975 .

[29]  M. Rosenblum,et al.  Human emotion recognition from motion using a radial basis function network architecture , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.