Are You Really Smiling at Me? Spontaneous versus Posed Enjoyment Smiles

Smiling is an indispensable element of nonverbal social interaction. Besides, automatic distinction between spontaneous and posed expressions is important for visual analysis of social signals. Therefore, in this paper, we propose a method to distinguish between spontaneous and posed enjoyment smiles by using the dynamics of eyelid, cheek, and lip corner movements. The discriminative power of these movements, and the effect of different fusion levels are investigated on multiple databases. Our results improve the state-of-the-art. We also introduce the largest spontaneous/posed enjoyment smile database collected to date, and report new empirical and conceptual findings on smile dynamics. The collected database consists of 1240 samples of 400 subjects. Moreover, it has the unique property of having an age range from 8 to 76 years. Large scale experiments on the new database indicate that eyelid dynamics are highly relevant for smile classification, and there are age-related differences in smile dynamics.

[1]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[2]  P. Ekman,et al.  The symmetry of emotional and deliberate facial actions. , 1981, Psychophysiology.

[3]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[4]  P. Ekman.  Telling lies: clues to deceit in the marketplace , 1985 .

[5]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Karen L. Schmidt,et al.  Signal characteristics of spontaneous facial expressions: automatic movement in solitary and social smiles , 2003, Biological Psychology.

[7]  Jing Xiao,et al.  Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[8]  Jeffrey F. Cohn,et al.  The Timing of Facial Motion in posed and Spontaneous Smiles , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[9]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  J. Cohn,et al.  Movement Differences between Deliberate and Spontaneous Facial Expressions: Zygomaticus Major Action in Smiling , 2006, Journal of nonverbal behavior.

[11]  Hatice Gunes,et al.  How to distinguish posed from spontaneous smiles using geometric features , 2007, ICMI '07.

[12]  J. Cohn,et al.  All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous , 2009, Journal of nonverbal behavior.

[13]  A. Manstead,et al.  Can Duchenne smiles be feigned? New evidence on felt and false smiles. , 2009, Emotion.

[14]  Karen L. Schmidt,et al.  Comparison of Deliberate and Spontaneous Facial Movement in Smiles and Eyebrow Raises , 2009, Journal of nonverbal behavior.

[15]  Fei Chen,et al.  A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference , 2010, IEEE Transactions on Multimedia.

[16]  Albert Ali Salah,et al.  Eyes do not lie: spontaneous versus posed smiles , 2010, ACM Multimedia.

[17]  M. Pantic,et al.  Induced Disgust , Happiness and Surprise : an Addition to the MMI Facial Expression Database , 2010 .

[18]  M. Del Giudice,et al.  Individual Differences in the Recognition of Enjoyment Smiles: No Role for Perceptual–Attentional Factors and Autistic-Like Traits , 2011, Front. Psychology.

[19]  Matti Pietikäinen,et al.  Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[20]  Punit Shah.  Toward a Neurobiology of Unrealistic Optimism , 2012, Front. Psychology.

[21]  Albert Ali Salah,et al.  A smile can reveal your age: enabling facial dynamics in age estimation , 2012, ACM Multimedia.

[22]  Albert Ali Salah,et al.  A Statistical Method for 2-D Facial Landmarking , 2012, IEEE Transactions on Image Processing.