Expert system for automatic analysis of facial expressions

Abstract This paper discusses our expert system called Integrated System for Facial Expression Recognition (ISFER), which performs recognition and emotional classification of human facial expression from a still full-face image. The system consists of two major parts. The first one is the ISFER Workbench, which forms a framework for hybrid facial feature detection. Multiple feature detection techniques are applied in parallel. The redundant information is used to define unambiguous face geometry containing no missing or highly inaccurate data. The second part of the system is its inference engine called HERCULES, which converts low level face geometry into high level facial actions, and then this into highest level weighted emotion labels.

[1]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[2]  Joseph O'Rourke,et al.  Computational Geometry in C. , 1995 .

[3]  C. Izard.  The face of emotion , 1971 .

[4]  O. Nakamura,et al.  Description and synthesis of facial expression based on isodensity maps , 1992 .

[5]  Mubarak Shah,et al.  A Fast algorithm for active contours and curvature estimation , 1992, CVGIP Image Underst..

[6]  Takeo Kanade,et al.  Feature-point tracking by optical flow discriminates subtle differences in facial expression , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[7]  A. Anderson,et al.  The Effects of Visibility on Dialogue and Performance in a Cooperative Problem Solving Task , 1994 .

[8]  Léon J. M. Rothkrantz,et al.  Mixed Fuzzy-system and Artificial Neural Network Approach to the Automated Recognition of Mouth Expressions , 1998 .

[9]  J. N. Bassili.  Facial motion in the perception of faces and of emotional expression. , 1978, Journal of experimental psychology. Human perception and performance.

[10]  Takeo Kanade,et al.  Optical flow estimation using wavelet motion model , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  S. Demleitner.  [Communication without words]. , 1997, Pflege aktuell.

[12]  Pat Hanrahan,et al.  Visual Computing , 2000, CG International Series.

[13]  Sati McKenzie,et al.  Machine Interpretation of Emotion: Design of a Memory-Based Expert System for Interpreting Facial Expressions in Terms of Signaled Emotions , 1993, Cogn. Sci..

[14]  Takeo Kanade,et al.  Automated facial expression recognition based on FACS action units , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[15]  Shigeo Morishima,et al.  A modeling of facial expression and emotion for recognition and synthesis , 1995 .

[16]  Joseph N. Wilson,et al.  Handbook of computer vision algorithms in image algebra , 1996 .

[17]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[18]  A. Wojdel,et al.  Analysis of Facial Expressions Based on Silhouettes , 1999 .

[19]  Andrew S. Glassner,et al.  Graphics Gems , 1990 .

[20]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[22]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Prem Kalra,et al.  Face to virtual face , 1998, Proc. IEEE.

[24]  P. Ekman.  Unmasking The Face , 1975 .

[25]  M. K. Khan,et al.  Machine identification of human faces , 1981, Pattern Recognition.

[26]  H. Yamada.  Visual Information for Categorizing Facial Expression of Emotions , 1993 .

[27]  P. Ekman.  Emotion in the human face , 1982 .

[28]  A. J. Fridlund.  IS THERE UNIVERSAL RECOGNITION OF EMOTION FROM FACIAL EXPRESSION? A REVIEW OF THE CROSS-CULTURAL STUDIES , 1994 .

[29]  G. Kearney,et al.  Machine Interpretation of Emotion: Design of a Memory‐Based Expert System for Interpreting Facial Expressions in Terms of Signaled Emotions , 1993 .

[30]  Emanuele Trucco,et al.  Computer and Robot Vision , 1995 .

[31]  Hidenori Itoh,et al.  Image Filtering, Edge Detection, and Edge Tracing Using Fuzzy Reasoning , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  F. Hara,et al.  Facial interaction between animated 3D face robot and human beings , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[33]  D. Rutter,et al.  The Role of Visual Communication in Social Exchange , 1976 .

[34]  J. Russell.  Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. , 1994, Psychological bulletin.

[35]  Shigeo Morishima,et al.  Emotion space for analysis and synthesis of facial expression , 1993, Proceedings of 1993 2nd IEEE International Workshop on Robot and Human Communication.