Recognition of affective prosody: continuous wavelet measures of event-related brain potentials to emotional exclamations.

The affective state of a speaker can be identified from the prosody of his or her speech. Voice quality is the most important prosodic cue for emotion recognition from short verbal utterances and nonverbal exclamations, the latter conveying pure emotion, void of all semantic meaning. We adopted two context violation paradigms-oddball and priming-to study the event-related brain potentials (ERP) reflecting this recognition process. We found a negative wave, the N300, in the ERPs to contextually incongruous exclamations, and interpreted this component as analogous to the well-known N400 response to semantically inappropriate words. The N300 appears to be a real-time psychophysiological measure of spontaneous emotion recognition from vocal cues, which could prove a useful tool for the examination of affective-prosody comprehension. In addition, we developed a new ERP component detection and estimation method that is based on the continuous wavelet transform (CWT), does not rely on visual inspection of the waveforms, and yields larger statistical difference effects than classical methods.

[1]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[2]  Irwin Pollack,et al.  Communication of Verbal Modes of Expression , 1960 .

[3]  David Crystal,et al.  Prosodic Systems and Intonation in English , 1969 .

[4]  D. Premack,et al.  Language in chimpanzee? , 1971, Science.

[5]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[6]  P. Marler,et al.  Monkey responses to three different alarm calls: evidence of predator classification and semantic communication. , 1980, Science.

[7]  J. Laver.  The phonetic description of voice quality , 1980 .

[8]  E. Ross,et al.  The aprosodias. Functional-anatomic organization of the affective components of language in the right hemisphere. , 1981, Archives of neurology.

[9]  E Donchin,et al.  A new method for off-line removal of ocular artifact. , 1983, Electroencephalography and clinical neurophysiology.

[10]  S. Farmer,et al.  The effects of physical and semantic incongruities on auditory event-related potentials. , 1984, Electroencephalography and clinical neurophysiology.

[11]  Klaus R. Scherer,et al.  Vocal Affect Signaling: A Comparative Approach , 1985 .

[12]  C. C. Wood,et al.  Event-related potentials, lexical decision and semantic priming. , 1985, Electroencephalography and clinical neurophysiology.

[13]  K. Scherer.  Vocal affect expression: a review and a model for future research. , 1986, Psychological bulletin.

[14]  Arvid Kappas,et al.  Primate Vocal Expression of Affective State , 1988 .

[15]  Gregory A. Miller,et al.  Generalized Implementation of an Eye Movement Correction Procedure , 1988 .

[16]  S. Folstein,et al.  Comprehension of prosody in Huntington's disease. , 1990, Journal of neurology, neurosurgery, and psychiatry.

[17]  S. Woodward,et al.  Word-to-word variation in ERP component latencies: Spoken words , 1990, Brain and Language.

[18]  Reijo Aulanko,et al.  Vocal Communication between Species: Man and Macaque. , 1991 .

[19]  Event-related potentials in disorders of prosodic and semantic linguistic processing 1 , 1991 .

[20]  R. Erwin,et al.  P3 responses to prosodic stimuli in adult autistic subjects. , 1991, Electroencephalography and clinical neurophysiology.

[21]  A. Groothuis.  Advances in the Study of Behavior , 1993 .

[22]  Joseph E. LeDoux,et al.  Emotional memory systems in the brain , 1993, Behavioural Brain Research.

[23]  M. Kutas,et al.  Electrophysiological evidence for task effects on semantic priming in auditory word processing. , 1993, Psychophysiology.

[24]  A Kok,et al.  A comparison of different methods for estimating single-trial P300 latencies. , 1994, Electroencephalography and clinical neurophysiology.

[25]  R. Adolphs,et al.  Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala , 1994, Nature.

[26]  I. Linnankoski,et al.  Recognition of emotions in macaque vocalizations by children and adults , 1994 .

[27]  C. Petten,et al.  Conceptual relationships between spoken words and environmental sounds: Event-related brain potential measures , 1995, Neuropsychologia.

[28]  R. Lane,et al.  Impaired Verbal and Nonverbal Emotion Recognition in Alexithymia , 1996, Psychosomatic medicine.

[29]  Colin M. Brown,et al.  Lexical-semantic event-related potential effects in patients with left hemisphere lesions and aphasia, and patients with right hemisphere lesions without aphasia. , 1996, Brain : a journal of neurology.

[30]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[31]  I. Linnankoski,et al.  Expression or emotional-motivational connotations with a one-word utterance. , 1997, The Journal of the Acoustical Society of America.

[32]  E. Ross,et al.  Lateralization of Affective Prosody in Brain and the Callosal Integration of Hemispheric Language Functions , 1997, Brain and Language.

[33]  E Altenmüller,et al.  The cortical processing of perceived emotion: a DC‐potential study on affective speech prosody , 1997, Neuroreport.

[34]  F. Schlaghecken.  On Processing BEASTS and BIRDS: An Event-Related Potential Study on the Representation of Taxonomic Structure , 1998, Brain and Language.

[35]  R. M. Siegel,et al.  Lack of selectivity for simple shapes defined by motion and luminance in STPa of the behaving macaque , 1998, Neuroreport.

[36]  Hermann Ackermann,et al.  Emotional processing following cortical and subcortical brain damage: contribution of the fronto-striatal circuitry. , 1998, Behavioural neurology.

[37]  Peter Maass,et al.  EEG Signal Analysis by Continuous Wavelet Transform Techniques , 1998 .

[38]  Craig E. Tenke,et al.  Identification of Emotion in a Dichotic Listening Task: Event-Related Brain Potential and Behavioral Findings , 1998, Brain and Cognition.

[39]  A K Anderson,et al.  Intact recognition of vocal expressions of fear following bilateral lesions of the human amygdala. , 1998, Neuroreport.

[40]  Jeff T. Larsen,et al.  Negative information weighs more heavily on the brain: the negativity bias in evaluative categorizations. , 1998, Journal of personality and social psychology.

[41]  J. Pernier,et al.  ERP Manifestations of Processing Printed Words at Different Psycholinguistic Levels: Time Course and Scalp Distribution , 1999, Journal of Cognitive Neuroscience.

[42]  V. Samar,et al.  Wavelet Analysis of Neuroelectric Waveforms: A Conceptual Tutorial , 1999, Brain and Language.

[43]  Angela D. Friederici,et al.  Brain potentials indicate immediate use of prosodic cues in natural speech processing , 1999, Nature Neuroscience.

[44]  D. Everhart,et al.  Investigation of receptive affective prosodic ability in school-aged boys with and without depression. , 1999, Neuropsychiatry, neuropsychology, and behavioral neurology.

[45]  B Renault,et al.  Event-related potentials to structural familiar face incongruity processing. , 1999, Psychophysiology.

[46]  Daniel Tranel,et al.  Intact recognition of emotional prosody following amygdala damage , 1999, Neuropsychologia.

[47]  P. Holcomb,et al.  An electrophysiological investigation of semantic priming with pictures of real objects. , 1999, Psychophysiology.

[48]  R. Tannock,et al.  Dichotic listening and response inhibition in children with comorbid anxiety disorders and ADHD. , 2000, Journal of the American Academy of Child and Adolescent Psychiatry.

[49]  E Altenmüller,et al.  Cortical activation patterns of affective speech processing depend on concurrent demands on the subvocal rehearsal system. A DC-potential study. , 2000, Brain : a journal of neurology.

[50]  R. Buck,et al.  Affective-prosodic deficits in schizophrenia: profiles of patients with brain damage and comparison with relation to schizophrenic symptoms , 2001 .

[51]  E. Ross,et al.  Altered emotional perception in alcoholics: deficits in affective prosody comprehension. , 2001, Alcoholism, clinical and experimental research.

[52]  B. Kotchoubey,et al.  Event-related potentials in an auditory semantic oddball task in humans , 2001, Neuroscience Letters.

[53]  Kara D. Federmeier,et al.  Switching Languages, Switching Palabras (Words): An Electrophysiological Study of Code Switching , 2002, Brain and Language.

[54]  Vladimir Bostanov,et al.  BCI competition 2003-data sets Ib and IIb: feature extraction from event-related brain potentials with the continuous wavelet transform and the t-value scalogram , 2004, IEEE Transactions on Biomedical Engineering.