Facial expression recognition from line-based caricatures

The automatic recognition of facial expression presents a significant challenge to the pattern analysis and man-machine interaction research community. Recognition from a single static image is particularly a difficult task. In this paper, we present a methodology for facial expression recognition from a single static image using line-based caricatures. The recognition process is completely automatic. It also addresses the computational expensive problem and is thus suitable for real-time applications. The proposed approach uses structural and geometrical features of a user sketched expression model to match the line edge map (LEM) descriptor of an input face image. A disparity measure that is robust to expression variations is defined. The effectiveness of the proposed technique has been evaluated and promising results are obtained. This work has proven the proposed idea that facial expressions can be characterized and recognized by caricatures.

[1]  Hong Yan,et al.  Locating and extracting the eye in human face images , 1996, Pattern Recognit..

[2]  Yongsheng Gao,et al.  Face Recognition Using Line Edge Map , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Garrison W. Cottrell,et al.  EMPATH: Face, Emotion, and Gender Recognition Using Holons , 1990, NIPS.

[4]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[5]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Thomas O. Binford,et al.  On Detecting Edges , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[8]  Kyu Ho Park,et al.  Automatic human face location in a complex background using motion and color information , 1996, Pattern Recognit..

[9]  Jian-Gang Wang,et al.  Frontal-view face detection and facial feature extraction using color and morphological operations , 1999, Pattern Recognit. Lett..

[10]  P. Ekman.  Unmasking The Face , 1975 .

[11]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[12]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[14]  Barnabás Takács,et al.  Comparing face images using the modified Hausdorff distance , 1998, Pattern Recognit..

[15]  Maja Pantic,et al.  Expert system for automatic analysis of facial expressions , 2000, Image Vis. Comput..

[16]  Sudeep Sarkar,et al.  Robust Visual Method for Assessing the Relative Performance of Edge-Detection Algorithms , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[18]  Zhengyou Zhang,et al.  Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with A Multilayer Perceptron , 1999, Int. J. Pattern Recognit. Artif. Intell..

[19]  Kim L. Boyer,et al.  Assessing the state of the art in edge detection: 1992 , 1992, Defense, Security, and Sensing.

[20]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[21]  Larry S. Davis,et al.  Recognizing Human FACIAL EXPRESSION , 1994 .

[22]  Sudeep Sarkar,et al.  Comparison of Edge Detectors: A Methodology and Initial Study , 1998, Comput. Vis. Image Underst..

[23]  J. Mundy,et al.  Driving vision by topology , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[24]  James Jenn-Jier Lien,et al.  A Multi-Method Approach for Discriminating Between Similar Facial Expressions, Including Expression Intensity Estimation , 1998 .

[25]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Beat Fasel,et al.  Recognition of asymmetric facial action unit activities and intensities , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[28]  Garrison W. Cottrell,et al.  Representing Face Images for Emotion Classification , 1996, NIPS.

[29]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[30]  Fredrik Bergholm,et al.  Edge Focusing , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[32]  Yee-Hong Yang,et al.  Dynamic two-strip algorithm in curve fitting , 1990, Pattern Recognit..

[33]  A. Rogier.  [Communication without words]. , 1971, Tijdschrift voor ziekenverpleging.

[34]  Irwin Edward Sobel,et al.  Camera Models and Machine Perception , 1970 .

[35]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[37]  Sudeep Sarkar,et al.  Comparison of edge detectors: a methodology and initial study , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  David I. Perrett,et al.  Self priming from distinctive and caricatured faces , 1996 .

[40]  K. Ramesh Babu,et al.  Linear Feature Extraction and Description , 1979, IJCAI.

[41]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Alex Pentland,et al.  LAFTER: a real-time face and lips tracker with facial expression recognition , 2000, Pattern Recognit..

[43]  Steven W. Zucker,et al.  Logical/Linear Operators for Image Curves , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Gang Chen,et al.  Edge detection by regularized cubic B-spline fitting , 1995, IEEE Trans. Syst. Man Cybern..

[46]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..