A Comparative Study of Alternative FACS Coding Algorithms

Two groups were contracted to experiment with coding of FACS (Ekman & Friesen, 1978) action units on a common database. One group is ours at CMU and the University of Pittsburgh, and the other is at UCSD. The database is from Frank and Ekman (1997) who video-recorded an interrogation in which subjects lied or told the truth about a mock crime. Subjects were ethnically diverse, action units occurred during speech, and out-of-plane motion and occlusion from head motion and glasses were common. The video data were originally collected to answer substantive questions in psychology, and represent a substantial challenge to automated AU recognition. This report describes the results of automated facial expression analysis by the CMU/Pittsburgh group. An interdisciplinary team of consultants, who have combined expertise in computer vision and in facial analysis, will compare the results of this report with those in a separate report submitted by UCSD group.

[1]  P. Ekman,et al.  Approaches To Emotion , 1985 .

[2]  P. Ekman,et al.  Behavioral Markers and Recognizability of the Smile of Enjoyment , 1993 .

[3]  O. Blin,et al.  Apomorphine-induced blinking and yawning in healthy volunteers. , 1990, British journal of clinical pharmacology.

[4]  G. Cottrell,et al.  Categorical Perception in Facial Emotion Classification , 1996 .

[5]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  P. Ekman,et al.  The ability to detect deceit generalizes across different types of high-stake lies. , 1997, Journal of personality and social psychology.

[7]  Marian Stewart Bartlett,et al.  Automatic Analysis of Spontaneous Facial Behavior: A Final Project Report , 2001 .

[8]  P. Ekman,et al.  Behavioral markers and recognizability of the smile of enjoyment. , 1993, Journal of personality and social psychology.

[9]  Jeffrey F. Cohn,et al.  Dynamics of facial expression: normative characteristics and individual differences , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[10]  A. J. Fridlund.  Human Facial Expression: An Evolutionary View , 1994 .

[11]  Alex Pentland,et al.  A vision system for observing and extracting facial action parameters , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  L. Farkas,et al.  Anthropometric Facial Proportions in Medicine , 1986 .

[13]  P. Ekman.  Expression and the Nature of Emotion , 1984 .

[14]  David J. Fleet,et al.  Learning parameterized models of image motion , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  T. Sejnowski,et al.  Measuring facial expressions by computer image analysis. , 1999, Psychophysiology.

[16]  W. Rinn,et al.  The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions. , 1984, Psychological bulletin.

[17]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  J. Cohn,et al.  Specific Impairment of Smiling Increases the Severity of Depressive Symptoms in Patients with Facial Neuromuscular Disorders , 1999, Aesthetic Plastic Surgery.

[19]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[20]  Karson Cn.  Physiology of normal and abnormal blinking. , 1988 .

[21]  J. Cacioppo,et al.  Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. , 1986, Journal of personality and social psychology.

[22]  Karen L. Schmidt,et al.  Human facial expressions as adaptations: Evolutionary questions in facial expression research. , 2001, American journal of physical anthropology.

[23]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[24]  N L Foster,et al.  Blink rates and disorders of movement , 1984, Neurology.

[25]  D. Barash.  Human Ethology , 1973 .

[26]  C. Darwin,et al.  The Expression of the Emotions in Man and Animals , 1872 .

[27]  J. Cohn,et al.  Psychological Distress , 1998, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[28]  M. Holland,et al.  Blinking and Mental Load , 1972, Psychological reports.

[29]  Michael J. Black,et al.  Recognizing facial expressions under rigid and non-rigid facial motions , 1995 .

[30]  B. Everitt,et al.  Statistical methods for rates and proportions , 1973 .

[31]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[32]  D R MEYER,et al.  Incentive, anxiety, and the human blink rate. , 1953, Journal of experimental psychology.

[33]  P. Ekman,et al.  EMFACS-7: Emotional Facial Action Coding System , 1983 .

[34]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  P. Gosselin,et al.  Components and recognition of facial expression in the communication of emotion by actors. , 1995, Journal of personality and social psychology.

[36]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  R. Depue,et al.  Dopamine and the structure of personality: relation of agonist-induced dopamine activity to positive emotionality. , 1994, Journal of personality and social psychology.

[38]  Takeo Kanade,et al.  Detection, tracking, and classification of action units in facial expression , 2000, Robotics Auton. Syst..

[39]  L. Camras,et al.  Infant “surprise” expressions as coordinative motor structures , 1996 .

[40]  Klaus R. Scherer,et al.  What does facial expression express , 1992 .