The vocal communication of different kinds of smile

The present study investigated the vocal communication of naturally occurring smiles. Verbal variation was controlled in the speech of 8 speakers by asking them to repeat the same sentence in response to a set sequence of 17 questions, intended to provoke reactions such as amusement, mild embarrassment, or just a neutral response. After coding for facial expressions, a sample of 64 utterances was chosen to represent Duchenne smiles, non-Duchenne smiles, suppressed smiles and non-smiles. These audio clips were used to test the discrimination skills of 11 listeners, who had to rely on vocal indicators to identify different types of smiles in speech. The study established that listeners can discriminate different smile types and further indicated that listeners utilize prototypical ideals to discern whether a person is smiling. Some acoustical cues appear to be taken by listeners as strong indicators of a smile, regardless of whether the speaker is actually smiling. Further investigations into listeners' prototypical ideals of vocal expressivity could prove worthwhile for voice synthesizing technology endeavoring to make computer-simulations more naturalistic.

[1]  M. Lindquist,et al.  A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions , 2002, Pharmacoepidemiology and drug safety.

[2]  P. Laukka,et al.  Impact of intended emotion intensity on cue utilization and decoding accuracy in vocal expression of emotion. , 2001, Emotion.

[3]  A. Manstead,et al.  Relations between emotions, display rules, social motives, and facial behaviour , 2004, Cognition & emotion.

[4]  K. Scherer,et al.  Evidence for universality and cultural variation of differential emotion response patterning. , 1994, Journal of personality and social psychology.

[5]  José-Miguel Fernández-Dols,et al.  Spontaneous facial behavior during intense emotional episodes: Artistic truth and optical truth. , 1997 .

[6]  Klaus R. Scherer,et al.  Vocal Affect Signaling: A Comparative Approach , 1985 .

[7]  P. Ladefoged.  A course in phonetics , 1975 .

[8]  J. Gross.  Antecedent- and response-focused emotion regulation: divergent consequences for experience, expression, and physiology. , 1998, Journal of personality and social psychology.

[9]  K. Scherer,et al.  Experiencing emotion : a cross-cultural study , 1986 .

[10]  W. Fitch.  Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques. , 1997, The Journal of the Acoustical Society of America.

[11]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[12]  Klaus R. Scherer,et al.  The role of intonation in emotional expressions , 2005, Speech Commun..

[13]  D. Keltner,et al.  Embarrassment: its distinct form and appeasement functions. , 1997, Psychological bulletin.

[14]  Stephen Malloch,et al.  The Dance of Wellbeing: Defining the Musical Therapeutic Effect , 2000 .

[15]  Véronique Aubergé,et al.  Can we hear the prosody of smile? , 2003, Speech Commun..

[16]  D. Keltner.  Signs of appeasement: evidence for the distinct displays of embarrassment, amusement, and shame , 1995 .

[17]  W. Morris,et al.  Toward the self-regulation of mood: Theory and research , 1987 .

[18]  K. Scherer,et al.  Children's ability to control the facial expression of laughter and smiling: Knowledge and behaviour , 2003, Cognition & emotion.

[19]  Klaus R. Scherer,et al.  Vocal communication of emotion , 2000 .

[20]  P. Philippot,et al.  The social context of nonverbal behavior , 1999 .

[21]  J. Swets.  Indices of discrimination or diagnostic accuracy: their ROCs and implied models. , 1986, Psychological bulletin.

[22]  D. Levitin.  Absolute memory for musical pitch: Evidence from the production of learned melodies , 1994, Perception & psychophysics.

[23]  P. Ekman,et al.  Facial signs of emotional experience. , 1980 .

[24]  J. Vroomen,et al.  The perception of emotions by ear and by eye , 2000 .

[25]  A. J. Fridlund.  Human Facial Expression: An Evolutionary View , 1994 .

[26]  Roddy Cowie,et al.  Speech and Emotion , 2003, Speech Commun..

[27]  J. Ohala,et al.  An Ethological Perspective on Common Cross-Language Utilization of F₀ of Voice , 1984, Phonetica.

[28]  Kim E. A. Silverman,et al.  Evidence for the independent function of intonation contour type, voice quality, and F0 range in signaling speaker affect , 1985 .

[29]  V C Tartter,et al.  Hearing smiles and frowns in normal and whisper registers. , 1994, The Journal of the Acoustical Society of America.

[30]  A. J. Fridlund.  Sociality of Solitary Smiling: Potentiation by an Implicit Audience , 1991 .

[31]  K. Scherer,et al.  Emotion Inferences from Vocal Expression Correlate Across Languages and Cultures , 2001 .

[32]  Travis Thompson,et al.  Behavioral observation : technology and applications in developmental disabilities , 2000 .

[33]  E. Tulving,et al.  Contingent dissociation between recognition and fragment completion: the method of triangulation. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[34]  Roddy Cowie,et al.  Describing the emotional states that are expressed in speech , 2003, Speech Commun..

[35]  V. Tartter.  Happy talk: Perceptual and acoustic effects of smiling on speech , 1980, Perception & psychophysics.

[36]  P. Ekman.  Telling lies: clues to deceit in the marketplace , 1985 .

[37]  P. Boersma.  ACCURATE SHORT-TERM ANALYSIS OF THE FUNDAMENTAL FREQUENCY AND THE HARMONICS-TO-NOISE RATIO OF A SAMPLED SOUND , 1993 .

[38]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[39]  Daniel S. Messinger,et al.  The psychology of facial expression: A dynamic systems approach to infant facial action , 1997 .

[40]  Sheldon B. Michaels,et al.  Some Aspects of Fundamental Frequency and Envelope Amplitude as Related to the Emotional Content of Speech , 1962 .

[41]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[42]  L. F. Barrett,et al.  Handbook of Emotions , 1993 .

[43]  Marianne LaFrance,et al.  Option or obligation to smile: The effects of power and gender on facial expression. , 1999 .

[44]  P. Ekman,et al.  Detecting deception from the body or face. , 1974 .

[45]  J. Russell,et al.  The psychology of facial expression: Frontmatter , 1997 .

[46]  Jennifer S. Beer,et al.  Facial expression of emotion. , 2003 .

[47]  Klaus R. Scherer,et al.  Vocal communication of emotion: A review of research paradigms , 2003, Speech Commun..

[48]  P. Ekman,et al.  Smiles when lying. , 1988, Journal of personality and social psychology.

[49]  Suthathip Chuenwattanapranithi,et al.  PERCEIVING ANGER AND JOY IN SPEECH THROUGH THE SIZE CODE , 2007 .

[50]  R. Frick.  Communicating emotion: The role of prosodic features. , 1985 .

[51]  P. Ekman,et al.  The nature of emotion: Fundamental questions. , 1994 .

[52]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[53]  J. M. Bevan,et al.  Rank Correlation Methods , 1949 .