The Production and Perception of Emotionally Expressive Walking Sounds: Similarities between Musical Performance and Everyday Motor Activity

Several studies have investigated the encoding and perception of emotional expressivity in music performance. A relevant question concerns how the ability to communicate emotions in music performance is acquired. In accordance with recent theories on the embodiment of emotion, we suggest here that both the expression and recognition of emotion in music might at least in part rely on knowledge about the sounds of expressive body movements. We test this hypothesis by drawing parallels between musical expression of emotions and expression of emotions in sounds associated with a non-musical motor activity: walking. In a combined production-perception design, two experiments were conducted, and expressive acoustical features were compared across modalities. An initial performance experiment tested for similar feature use in walking sounds and music performance, and revealed that strong similarities exist. Features related to sound intensity, tempo and tempo regularity were identified as been used similarly in both domains. Participants in a subsequent perception experiment were able to recognize both non-emotional and emotional properties of the sound-generating walkers. An analysis of the acoustical correlates of behavioral data revealed that variations in sound intensity, tempo, and tempo regularity were likely used to recognize expressed emotions. Taken together, these results lend support the motor origin hypothesis for the musical expression of emotions.

[1]  Die Musik als Ausdruck , 1887 .

[2]  William B. Michael,et al.  Psychological Scaling: Theory and Applications , 1961 .

[3]  R. Iman,et al.  The Use of the Rank Transform in Regression , 1979 .

[4]  Stephen Davies The expression of emotion in music , 1980 .

[5]  J. Russell A circumplex model of affect. , 1980 .

[6]  K. Scherer Vocal affect expression: a review and a model for future research. , 1986, Psychological bulletin.

[7]  A. Thorstensson,et al.  Adaptability in frequency and amplitude of leg movements during human locomotion at different speeds. , 1987, Acta physiologica Scandinavica.

[8]  A. Thorstensson,et al.  Ground reaction forces at different speeds of human walking and running. , 1989, Acta physiologica Scandinavica.

[9]  Robert J. Logan,et al.  Perception of acoustic source characteristics: walking sounds. , 1991, The Journal of the Acoustical Society of America.

[10]  Johan Sundberg,et al.  Threshold and Preference Quantities of Rules for Music Performance , 1991 .

[11]  P. Juslin,et al.  Emotional Expression in Music Performance: Between the Performer's Intention and the Listener's Experience , 1996 .

[12]  Stephen T. Neely,et al.  Signals, Sound, and Sensation , 1997 .

[13]  P. Juslin Emotional Communication in Music Performance: A Functionalist Perspective and Some Data , 1997 .

[14]  P. Juslin Perceived Emotional Expression in Synthesized Performances of a Short Melody: Capturing the Listener's Judgment Policy , 1997 .

[15]  I. Peretz,et al.  Music and emotion: perceptual determinants, immediacy, and isolation after brain damage , 1998, Cognition.

[16]  J. Sundberg,et al.  Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping , 1999 .

[17]  Anders Friberg,et al.  Emotional Coloring of Computer-Controlled Music Performances , 2000, Computer Music Journal.

[18]  E. Gibson,et al.  An Ecological Approach to Perceptual Learning and Development , 2000 .

[19]  Joshua D. Naranjo,et al.  A Robust Method for the Analysis of Experiments with Ordered Treatment Levels , 2001, Psychological reports.

[20]  Davide Rocchesso,et al.  The Sounding Object , 2002 .

[21]  Tracy Brown,et al.  The Embodied Mind: Cognitive Science and Human Experience , 2002, Cybern. Hum. Knowing.

[22]  G. Rizzolatti,et al.  Hearing Sounds, Understanding Actions: Action Representation in Mirror Neurons , 2002, Science.

[23]  Sofia Dahl,et al.  Experiments on gestures: walking, running, and hitting. , 2003 .

[24]  Scott T. Grafton,et al.  Swinging in the brain: shared neural substrates for behaviors related to sequencing and music , 2003, Nature Neuroscience.

[25]  Roberto Bresin,et al.  “Expressivity comes from within your soul”: A questionnaire study of music students' perspectives on expressivity , 2003 .

[26]  M. Bangert,et al.  Mapping perception to action in piano practice: a longitudinal DC-EEG study , 2003, BMC Neuroscience.

[27]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[28]  S. McAdams,et al.  The psychomechanics of simulated sound sources: material properties of impacted bars. , 2004, The Journal of the Acoustical Society of America.

[29]  J. Sloboda,et al.  Perception And Cognition Of Music , 2004 .

[30]  J. Marozeau,et al.  Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts , 2005 .

[31]  P. Juslin,et al.  Play it again with feeling: computer feedback in musical communication of emotions. , 2006, Journal of experimental psychology. Applied.

[32]  Hans-Jochen Heinze,et al.  Shared networks for auditory and motor processing in professional pianists: Evidence from fMRI conjunction , 2006, NeuroImage.

[33]  Istvan Molnar-Szakacs,et al.  Music and mirror neurons: from motion to 'e'motion. , 2006, Social cognitive and affective neuroscience.

[34]  A. Friederici,et al.  Investigating emotion with music: An fMRI study , 2006, Human brain mapping.

[35]  B. Gelder Towards the neurobiology of emotional body language , 2006, Nature Reviews Neuroscience.

[36]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[37]  R. Pastore,et al.  Auditory event perception: The source—perception loop for posture in human gait , 2008, Perception & psychophysics.

[38]  P. Janata,et al.  Embodied music cognition and mediation technology , 2009 .

[39]  Christian Keysers,et al.  Expanding the mirror: vicarious activity for actions, emotions, and sensations , 2009, Current Opinion in Neurobiology.

[40]  K. Scherer,et al.  Bodily expression of emotion , 2009 .

[41]  Tom Cochrane,et al.  A Simulation Theory of Musical Expressivity , 2010 .

[42]  Roberto Bresin,et al.  Emotion rendering in music: Range and characteristic values of seven musical variables , 2011, Cortex.