Nonlinguistic vocalizations from online amateur videos for emotion research: A validated corpus

This study introduces a corpus of 260 naturalistic human nonlinguistic vocalizations representing nine emotions: amusement, anger, disgust, effort, fear, joy, pain, pleasure, and sadness. The recognition accuracy in a rating task varied greatly per emotion, from <40% for joy and pain, to >70% for amusement, pleasure, fear, and sadness. In contrast, the raters’ linguistic–cultural group had no effect on recognition accuracy: The predominantly English-language corpus was classified with similar accuracies by participants from Brazil, Russia, Sweden, and the UK/USA. Supervised random forest models classified the sounds as accurately as the human raters. The best acoustic predictors of emotion were pitch, harmonicity, and the spacing and regularity of syllables. This corpus of ecologically valid emotional vocalizations can be filtered to include only sounds with high recognition rates, in order to study reactions to emotional stimuli of known perceptual types (reception side), or can be used in its entirety to study the association between affective states and vocal expressions (production side).

[1]  H. Wagner.  On measuring performance in category judgment studies of nonverbal behavior , 1993 .

[2]  Sophie K. Scott,et al.  Laugh Like You Mean It: Authenticity Modulates Acoustic, Physiological and Perceptual Properties of Laughter , 2016 .

[3]  Sophie K. Scott,et al.  Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations , 2010, Proceedings of the National Academy of Sciences.

[4]  P. Belin,et al.  Cross-Cultural Differences in the Processing of Non-Verbal Affective Vocalizations by Japanese and Canadian Listeners , 2013, Front. Psychol..

[5]  S. Scott,et al.  When voices get emotional: A corpus of nonverbal vocalizations for research on emotion processing , 2013, Behavior research methods.

[6]  Cynthia Breazeal,et al.  Recognition of Affective Communicative Intent in Robot-Directed Speech , 2002, Auton. Robots.

[7]  J. Newman.  Neural circuits underlying crying and cry responding in mammals , 2007, Behavioural Brain Research.

[8]  Scherer,et al.  On the use of actor portrayals in research on emotional expression , 2010 .

[9]  Drew Rendall,et al.  Two organizing principles of vocal production: Implications for nonhuman and human primates , 2011, American journal of primatology.

[10]  Andreas Nieder,et al.  Cognitive Control of Distinct Vocalizations in Rhesus Monkeys , 2013, Journal of Cognitive Neuroscience.

[11]  G. Arriaga.  Why the Caged Mouse Sings: Studies of the Mouse Ultrasonic Song System and Vocal Behavior , 2014 .

[12]  Klaus R. Scherer,et al.  Vocal markers of emotion: Comparing induction and acting elicitation , 2013, Comput. Speech Lang..

[13]  R. Schusterman,et al.  3Vocal Learning in Mammals with Special Emphasis on Pinnipeds , 2008 .

[14]  P. Zelazo,et al.  What is Cognitive Control , 2013 .

[15]  S. Nowicki,et al.  The Evolution of Animal Communication: Reliability and Deception in Signaling Systems: Reliability and Deception in Signaling Systems , 2005 .

[16]  K. Scherer,et al.  Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception. , 2012, Emotion.

[17]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[18]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[19]  C. Athena Aktipis,et al.  The animal nature of spontaneous human laughter , 2014 .

[20]  L. F. Barrett,et al.  Cultural Relativity in Perceiving Emotion From Vocalizations , 2014, Psychological science.

[21]  Emiliana R. Simon-Thomas,et al.  The voice conveys specific emotions: evidence from vocal burst displays. , 2009, Emotion.

[22]  Petri Laukka,et al.  Cross-cultural decoding of positive and negative non-linguistic emotion vocalizations , 2013, Front. Psychol..

[23]  S. Scott,et al.  Perceptual Cues in Nonverbal Vocal Expressions of Emotion , 2010 .

[24]  F. Gosselin,et al.  The Montreal Affective Voices: A validated set of nonverbal affect bursts for research on auditory affective processing , 2008, Behavior research methods.

[25]  K. Hammerschmidt,et al.  Characterizing Vocal Repertoires—Hard vs. Soft Classification Approaches , 2015, PloS one.

[26]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[27]  S. Brudzyński.  Social Origin of Vocal Communication in Rodents , 2014 .

[28]  Sonja A. Kotz,et al.  Factors in the recognition of vocally expressed emotions: A comparison of four languages , 2009, J. Phonetics.

[29]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[30]  Kirsten M Bohn,et al.  Syllable acoustics, temporal patterns, and call composition vary with behavioral context in Mexican free-tailed bats. , 2008, The Journal of the Acoustical Society of America.

[31]  Sophie K. Scott,et al.  Brain mechanisms for processing perceived emotional vocalizations in humans , 2009 .

[32]  Daniel Neiberg,et al.  Intra-, Inter-, and Cross-Cultural Classification of Vocal Affect , 2011, INTERSPEECH.

[33]  Marina Davila Ross,et al.  Reconstructing the Evolution of Laughter in Great Apes and Humans , 2009, Current Biology.

[34]  C. Boesch,et al.  Wild Chimpanzees Produce Group‐Specific Calls: a Case for Vocal Learning? , 2004 .

[35]  Julia Fischer,et al.  Encoding Conditions Affect Recognition of Vocally Expressed Emotions Across Cultures , 2013, Front. Psychol..

[36]  P. Ekman,et al.  The Duchenne smile: emotional expression and brain physiology. II. , 1990, Journal of personality and social psychology.

[37]  B. Horwitz,et al.  Laryngeal Motor Cortex and Control of Speech in Humans , 2011, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[38]  U. Jürgens,et al.  The neural control of vocalization in mammals: a review. , 2009, Journal of voice : official journal of the Voice Foundation.

[39]  P. Ekman.  An argument for basic emotions , 1992 .

[40]  Marc Schröder,et al.  Experimental study of affect bursts , 2003, Speech Commun..

[41]  C. Petkov,et al.  Birds, primates, and spoken language origins: behavioral phenotypes and neurobiological substrates , 2012, Front. Evol. Neurosci..

[42]  K. Fischer,et al.  DESPERATELY SEEKING EMOTIONS OR: ACTORS, WIZARDS, AND HUMAN BEINGS. , 2000 .

[43]  K. Scherer,et al.  Emotion Inferences from Vocal Expression Correlate Across Languages and Cultures , 2001 .

[44]  A. Christophe,et al.  Newborns' Cry Melody Is Shaped by Their Native Language , 2009, Current Biology.

[45]  J. Bachorowski,et al.  The acoustic features of human laughter. , 2001, The Journal of the Acoustical Society of America.

[46]  M. Craske,et al.  Introducing the Oxford Vocal (OxVoc) Sounds database: a validated set of non-acted affective sounds from human infants, adults, and domestic animals , 2014, Front. Psychol..

[47]  Elisabeth Scheiner,et al.  Vocal expression of emotions in normally hearing and hearing-impaired infants. , 2006, Journal of voice : official journal of the Voice Foundation.

[48]  Reijo Aulanko,et al.  Vocal Communication between Species: Man and Macaque. , 1991 .

[49]  S. Scott,et al.  More than one kind of happiness: Can we recognize vocal expressions of different positive states? , 2007 .

[50]  Roddy Cowie,et al.  Emotional speech: Towards a new generation of databases , 2003, Speech Commun..

[51]  Skyler T. Hawk,et al.  "Worth a thousand words": absolute and relative decoding of nonlinguistic affect vocalizations. , 2009, Emotion.

[52]  H. Barrett,et al.  Vocal Emotion Recognition Across Disparate Cultures , 2008 .

[53]  Hillary Anger Elfenbein,et al.  On the universality and cultural specificity of emotion recognition: a meta-analysis. , 2002, Psychological bulletin.