The Timing of Facial Motion in posed and Spontaneous Smiles

Almost all work in automatic facial expression analysis has focused on recognition of prototypic expressions rather than dynamic changes in appearance over time. To investigate the relative contribution of dynamic features to expression recognition, we used automatic feature tracking to measure the relation between amplitude and duration of smile onsets in spontaneous and deliberate smiles of 81 young adults of Euro- and African-American background. Spontaneous smiles were of smaller amplitude and had a larger and more consistent relation between amplitude and duration than deliberate smiles. A linear discriminant classifier using timing and amplitude measures of smile onsets achieved a 93% recognition rate. Using timing measures alone, recognition rate declined only marginally to 89%. These findings suggest that by extracting and representing dynamic as well as morphological features, automatic facial expression analysis can begin to discriminate among the message values of morphologically similar expressions.

[1]  R. Baloh,et al.  Quantitative measurement of saccade amplitude, duration, and velocity , 1975, Neurology.

[2]  Takeo Kanade,et al.  Detection, tracking, and classification of action units in facial expression , 2000, Robotics Auton. Syst..

[3]  Jing Xiao,et al.  Automatic recognition of eye blinking in spontaneously occurring behavior , 2002, Object recognition supported by user interaction for service robots.

[4]  Adena J. Zlochower,et al.  Vocal timing in face-to-face interaction of clinically depressed and nondepressed mothers and their 4-month-old infants , 1996 .

[5]  P. Richter,et al.  PSYCHOMETRIC PROPERTIES OF THE BECK DEPRESSION INVENTORY , 1994 .

[6]  J. Kehne.  The Neural Basis of Motor Control , 1987, The Yale Journal of Biology and Medicine.

[7]  L. Camras,et al.  Infant “surprise” expressions as coordinative motor structures , 1996 .

[8]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[9]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[10]  P. Ekman.  Universal facial expressions of emotion. , 1970 .

[11]  R. Scherer,et al.  Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS) , 2005 .

[12]  Heather Carmichael Olson,et al.  Substance-abusing women and their young infants: Early child outcome and sources of vulnerability , 1996 .

[13]  P. Ekman,et al.  Behavioral markers and recognizability of the smile of enjoyment. , 1993, Journal of personality and social psychology.

[14]  N. Frijda,et al.  Facial expression and modes of action readiness , 1997 .

[15]  Jeffrey F. Cohn,et al.  Dynamics of facial expression: normative characteristics and individual differences , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[16]  Jing Xiao,et al.  Robust full-motion recovery of head by dynamic templates and re-registration techniques , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[17]  J. Cohn,et al.  Automatic recognition of eye blinking in spontaneously occurring behavior , 2002 .

[18]  D. Keltner.  Signs of appeasement: evidence for the distinct displays of embarrassment, amusement, and shame , 1995 .

[19]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[20]  W. Rinn,et al.  The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions. , 1984, Psychological bulletin.

[21]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[22]  U. Dimberg,et al.  Rapid facial reactions to emotional facial expressions. , 1998, Scandinavian journal of psychology.

[23]  Karen L. Schmidt,et al.  Signal characteristics of spontaneous facial expressions: automatic movement in solitary and social smiles , 2003, Biological Psychology.

[24]  A. J. Fridlund.  Human Facial Expression: An Evolutionary View , 1994 .

[25]  J. Cohn,et al.  Specific Impairment of Smiling Increases the Severity of Depressive Symptoms in Patients with Facial Neuromuscular Disorders , 1999, Aesthetic Plastic Surgery.

[26]  Jing Xiao,et al.  Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[27]  N. Fox,et al.  Regional patterns of brain activity in adults with a history of childhood-onset depression: gender differences and clinical variability. , 2002, The American journal of psychiatry.

[28]  Daphne Blunt Bugental,et al.  Unmasking the "Polite Smile" , 1986 .

[29]  W. Rinn.  Neuropsychology of facial expression. , 1991 .

[30]  J. Cohn,et al.  Automated Tracking of Facial Features in Patients with Facial Neuromuscular Dysfunction , 2001, Plastic and reconstructive surgery.

[31]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  A. Beck,et al.  Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation , 1988 .

[33]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[34]  C. Darwin,et al.  The Expression of the Emotions in Man and Animals , 1872 .

[35]  Christiana M. Leonard,et al.  When's a Smile a Smile? Or how to Detect a Message by Digitizing the Signal , 1991 .