Covert digital manipulation of vocal emotion alter speakers’ emotional states in a congruent direction

Significance We created a digital audio platform to covertly modify the emotional tone of participants’ voices while they talked toward happiness, sadness, or fear. Independent listeners perceived the transformations as natural examples of emotional speech, but the participants remained unaware of the manipulation, indicating that we are not continuously monitoring our own emotional signals. Instead, as a consequence of listening to their altered voices, the emotional state of the participants changed in congruence with the emotion portrayed. This result is the first evidence, to our knowledge, of peripheral feedback on emotional experience in the auditory domain. This finding is of great significance, because the mechanisms behind the production of vocal emotion are virtually unknown. Research has shown that people often exert control over their emotions. By modulating expressions, reappraising feelings, and redirecting attention, they can regulate their emotional experience. These findings have contributed to a blurring of the traditional boundaries between cognitive and emotional processes, and it has been suggested that emotional signals are produced in a goal-directed way and monitored for errors like other intentional actions. However, this interesting possibility has never been experimentally tested. To this end, we created a digital audio platform to covertly modify the emotional tone of participants’ voices while they talked in the direction of happiness, sadness, or fear. The result showed that the audio transformations were being perceived as natural examples of the intended emotions, but the great majority of the participants, nevertheless, remained unaware that their own voices were being manipulated. This finding indicates that people are not continuously monitoring their own voice to make sure that it meets a predetermined emotional target. Instead, as a consequence of listening to their altered voices, the emotional state of the participants changed in congruence with the emotion portrayed, which was measured by both self-report and skin conductance level. This change is the first evidence, to our knowledge, of peripheral feedback effects on emotional experience in the auditory domain. As such, our result reinforces the wider framework of self-perception theory: that we often use the same inferential strategies to understand ourselves as those that we use to understand others.

[1]  M. Lorr,et al.  Profile of mood states , 1971 .

[2]  M. Inzlicht,et al.  Emotional foundations of cognitive control , 2015, Trends in Cognitive Sciences.

[3]  R. J Dolan,et al.  Activity in ventromedial prefrontal cortex covaries with sympathetic skin conductance level: a physiological account of a “default mode” of brain function , 2004, NeuroImage.

[4]  Avishai Henik,et al.  Cognitive strategies to regulate emotions—current evidence and future directions , 2014, Front. Psychol..

[5]  Nicolas Silvestrini,et al.  Mood effects on autonomic activity in mood regulation. , 2007, Psychophysiology.

[6]  Nathalie Malardier,et al.  Normes de concrétude, de valeur d'imagerie, de fréquence subjective et de valence émotionnelle pour 866 mots , 2003 .

[7]  K. Scherer,et al.  The New Handbook of Methods in Nonverbal Behavior Research , 2008 .

[8]  J. Gross Emotion Regulation: Current Status and Future Prospects , 2015 .

[9]  F. Strack,et al.  Inhibiting and facilitating conditions of the human smile: a nonobtrusive test of the facial feedback hypothesis. , 1988, Journal of personality and social psychology.

[10]  Agneta H. Fischer,et al.  Emotion response coherence: A dual-process perspective , 2014, Biological Psychology.

[11]  G. Pezzulo,et al.  The sensorimotor and social sides of the architecture of speech , 2014, Behavioral and Brain Sciences.

[12]  Steffen R. Hage,et al.  Comparative analyses of speech and language converge on birds , 2014, Behavioral and Brain Sciences.

[13]  Swann Pichon,et al.  Affective and Sensorimotor Components of Emotional Prosody Generation , 2013, The Journal of Neuroscience.

[14]  J. Lerner,et al.  Emotion and decision making. , 2015, Annual review of psychology.

[15]  James J. Gross,et al.  Emotion, Emotion Regulation, and Psychopathology , 2014 .

[16]  G. Lohmann,et al.  Interoceptive awareness changes the posterior insula functional connectivity profile , 2015, Brain Structure and Function.

[17]  T. Dalgleish,et al.  The emotional Stroop task and psychopathology. , 1996, Psychological bulletin.

[18]  M. Pickering,et al.  An integrated theory of language production and comprehension. , 2013, The Behavioral and brain sciences.

[19]  Sven C. Mueller,et al.  Probing emotional influences on cognitive control: an ALE meta-analysis of cognition emotion interactions , 2014, Brain Structure and Function.

[20]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[21]  E. Briefer Vocal Expression of Emotions in Mammals: Mechanisms of Production and Evidence , 2012 .

[22]  Tom Hollenstein,et al.  Models and methods of emotional concordance , 2014, Biological Psychology.

[23]  Hugo D. Critchley,et al.  Modulation of Emotional Appraisal by False Physiological Feedback during fMRI , 2007, PloS one.

[24]  W. Flack Peripheral feedback effects of facial expressions, bodily postures, and vocal expressions on emotional feelings , 2006 .

[25]  Maurice Lorr,et al.  Profile of Mood States (POMS) , 1989 .

[26]  A. Wohlschläger,et al.  The link between facial feedback and neural activity within central circuitries of emotion--new insights from botulinum toxin-induced denervation of frown muscles. , 2009, Cerebral cortex.

[27]  W. N. Schoenfeld,et al.  Principles of Psychology , 2007 .

[28]  Nick Chater,et al.  Choice Blindness and Preference Change: You Will Like This Paper Better If You (Believe You) Chose to Read It! , 2014 .

[29]  D. Keltner,et al.  Emotion-cognition interactions , 2015 .

[30]  Leonie Koban,et al.  Brain systems underlying the affective and social monitoring of actions: An integrative review , 2014, Neuroscience & Biobehavioral Reviews.

[31]  C. Larson,et al.  Voice F0 responses to manipulations in pitch feedback. , 1998, The Journal of the Acoustical Society of America.

[32]  Didier Grandjean,et al.  Functional neuroimaging of human vocalizations and affective speech , 2014, Behavioral and Brain Sciences.

[33]  Jeffery A. Jones,et al.  Perceptual calibration of F0 production: evidence from feedback perturbation. , 2000, The Journal of the Acoustical Society of America.

[34]  Guido Gainotti,et al.  Unconscious processing of emotions and the right hemisphere , 2012, Neuropsychologia.

[35]  J. Laird Self-attribution of emotion: the effects of expressive behavior on the quality of emotional experience. , 1974, Journal of personality and social psychology.

[36]  C. Batson,et al.  Value Judgments: Testing the Somatic-Marker Hypothesis Using False Physiological Feedback , 1999 .

[37]  P. Lang Behavioral treatment and bio-behavioral assessment: computer applications , 1980 .

[38]  C. Balkenius,et al.  Speakers’ Acceptance of Real-Time Speech Exchange Indicates That We Use Auditory Feedback to Specify the Meaning of What We Say , 2014, Psychological science.

[39]  D. Bem Self-Perception Theory , 1972 .

[40]  S. Asch The expression of emotions. , 1952 .

[41]  J. Laird,et al.  Bodily Influences on Emotional Feelings: Accumulating Evidence and Extensions of William James’s Theory of Emotion , 2014 .

[42]  Does it talk the talk? On the role of basal ganglia in emotive speech processing , 2014, Behavioral and Brain Sciences.

[43]  M. Guha APA Handbook of Personality and Social Psychology , 2015 .

[44]  M. Bradley,et al.  Looking at pictures: affective, facial, visceral, and behavioral reactions. , 1993, Psychophysiology.

[45]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[46]  J. Singer,et al.  Cognitive, social, and physiological determinants of emotional state. , 1962, Psychological review.

[47]  A. Seth Interoceptive inference, emotion, and the embodied self , 2013, Trends in Cognitive Sciences.

[48]  D. Grandjean,et al.  Peripheral responses to attended and unattended angry prosody: a dichotic listening paradigm. , 2011, Psychophysiology.

[49]  C. Darwin,et al.  The Expression of the Emotions in Man and Animals , 1872 .

[50]  Ralph Adolphs,et al.  The Biology of Fear , 2013, Current Biology.

[51]  H. Critchley,et al.  Knowing your own heart: Distinguishing interoceptive accuracy from interoceptive awareness , 2015, Biological Psychology.

[52]  D. Dennett The Intentional Stance. , 1987 .

[53]  K. LaBar Emotion–Cognition Interactions , 2010 .

[54]  A. Glenberg,et al.  Cosmetic Use of Botulinum Toxin-A Affects Processing of Emotional Language , 2009, Psychological science.

[55]  Greg Gibson,et al.  Rare and common variants: twenty arguments , 2012, Nature Reviews Genetics.

[56]  K. Scherer,et al.  Vocal expression of affect , 2005 .