The temporal connection between smiles and blinks

In this paper, we present evidence for a temporal relationship between eye blinks and smile dynamics (smile onset and offset). Smiles and blinks occur with high frequency during social interaction, yet little is known about their temporal integration. To explore the temporal relationship between them, we used an Active Appearance Models algorithm to detect eye blinks in video sequences that contained manually FACS-coded spontaneous smiles (AU 12). We then computed the temporal distance between blinks and smile onsets and offsets. Our data shows that eye blinks are correlated with the end of the smile and occur close to the offset, but before the lip corners stop moving downwards. Furthermore, a marginally significant effect suggests that eye blinks are suppressed (less frequent) before smile onset. For computer-generated characters, this model of the timing of blinks relative to smiles may be useful in creating compelling facial animations.

[1]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[2]  A. Bentivoglio,et al.  Analysis of blink rate patterns in normal subjects , 1997, Movement disorders : official journal of the Movement Disorder Society.

[3]  Karen L. Schmidt,et al.  Signal characteristics of spontaneous facial expressions: automatic movement in solitary and social smiles , 2003, Biological Psychology.

[4]  S. Porter,et al.  Reading Between the Lies , 2008, Psychological science.

[5]  C. Evinger,et al.  Eyelid movements. Mechanisms and normal data. , 1991, Investigative ophthalmology & visual science.

[6]  Heinrich H. Bülthoff,et al.  Evaluating the perceptual realism of animated facial expressions , 2008, TAP.

[7]  A. Manstead,et al.  Temporal Aspects of Facial Displays in Person and Expression Perception: The Effects of Smile Dynamics, Head-tilt, and Gender , 2007 .

[8]  Cynthia A. Stifter,et al.  The regulation of positive affect: Gaze aversion activity during mother-infant interaction , 1991 .

[9]  Jeffrey F. Cohn,et al.  The Timing of Facial Motion in posed and Spontaneous Smiles , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[10]  Takayuki Kanda,et al.  Footing in human-robot conversations: How robots might shape participant roles using gaze cues , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[12]  E. Ponder,et al.  ON THE ACT OF BLINKING , 1927 .

[13]  Dick F. Swaab,et al.  Sex differences in the effects of visual contact and eye contact in negotiations , 2009 .

[14]  Shigeru Kitazawa,et al.  Eyeblink entrainment at breakpoints of speech , 2010, Neuroscience Research.

[15]  Christopher E. Peters,et al.  Attention-driven eye gaze and blinking for virtual humans , 2003, SIGGRAPH '03.

[16]  M. Holland,et al.  Blinking and Thinking , 1975, Perceptual and motor skills.

[17]  Y. Tanaka,et al.  Blink Activity and Task Difficulty , 1993, Perceptual and motor skills.

[18]  C. Evinger,et al.  Blinking and associated eye movements in humans, guinea pigs, and rabbits. , 1984, Journal of neurophysiology.

[19]  M. Doughty.  Consideration of Three Types of Spontaneous Eyeblink Activity in Normal Humans: during Reading and Video Display Terminal Use, in Primary Gaze, and while in Conversation , 2001, Optometry and vision science : official publication of the American Academy of Optometry.

[20]  Iain Matthews,et al.  Modeling and animating eye blinks , 2011, TAP.

[21]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[22]  Karen L. Schmidt,et al.  Human facial expressions as adaptations: Evolutionary questions in facial expression research. , 2001, American journal of physical anthropology.

[23]  Jing Xiao,et al.  Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[24]  M. Bradley,et al.  Affective reactions to briefly presented pictures. , 2001, Psychophysiology.

[25]  J. Cohn,et al.  All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous , 2009, Journal of nonverbal behavior.

[26]  Hatice Gunes,et al.  How to distinguish posed from spontaneous smiles using geometric features , 2007, ICMI '07.

[27]  John P. Lewis,et al.  Automated eye motion using texture synthesis , 2005, IEEE Computer Graphics and Applications.

[28]  Jessica K. Hodgins,et al.  The saliency of anomalies in animated human characters , 2010, TAP.

[29]  A. J. Fridlund.  Sociality of Solitary Smiling: Potentiation by an Implicit Audience , 1991 .

[30]  Fred Cummins.  Gaze and blinking in dyadic conversation: A study in coordinated behaviour among individuals , 2012 .

[31]  A. Vrij,et al.  Blinking During and After Lying , 2008 .