How music touches: Musical parameters and listeners’ audio-tactile metaphorical mappings

Though the relationship of touch and sound is central to music performance, and audio-tactile metaphors are pertinent to musical discourse, few empirical studies have investigated systematically how musical parameters such as pitch height, loudness, timbre and their interactions affect auditory–tactile metaphorical mappings. In this study, 40 participants (20 musically trained) rated the appropriateness of six dichotomous tactile metaphors (sharp–blunt, smooth–rough, soft–hard, light–heavy, warm–cold and wet–dry) to 20 sounds varying in pitch height, loudness, instrumental timbre (violin vs. flute) and vibrato. Results (repeated measures MANOVA) suggest that tactile metaphors are strongly associated with all musical variables examined. For instance, higher pitches were rated as significantly sharper, rougher, harder, colder, drier and lighter than lower pitches. We consider several complementary accounts of the findings: psychophysical analogies between tactile and auditory sensory processing; experiential analogies, based on correlations between tactile and auditory qualities of sound sources in daily experience; and analogies based on abstract semantic dimensions, particularly potency and activity.

[1]  C. Spence,et al.  Audiotactile interactions in roughness perception , 2002, Experimental Brain Research.

[2]  John G. Neuhoff,et al.  The Doppler illusion: the influence of dynamic intensity change on perceived pitch. , 1996, Journal of experimental psychology. Human perception and performance.

[3]  S. S. Stevens.  The Volume and Intensity of Tones , 1934 .

[4]  D. Freed,et al.  Auditory correlates of perceived mallet hardness for a set of recorded percussive sound events. , 1990, The Journal of the Acoustical Society of America.

[5]  Riitta Hari,et al.  Touch activates human auditory cortex , 2006, NeuroImage.

[6]  Linda B. Smith,et al.  A developmental analysis of the polar structure of dimensions , 1992, Cognitive Psychology.

[7]  Renee Timmers,et al.  Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context , 2010, Cognition.

[8]  C Spence,et al.  The Role of Auditory Cues in Modulating the Perception of Electric Toothbrushes , 2003, Journal of dental research.

[9]  Charles E. Osgood,et al.  Semantic Differential Technique in the Comparative Study of Cultures1 , 2009 .

[10]  G. Deco,et al.  Multisensory contributions to the perception of vibrotactile events , 2009, Behavioural Brain Research.

[11]  Robert O. Gjerdingen,et al.  The psychology of music , 2002 .

[12]  S Smith,et al.  Stroop Interference Based on the Synaesthetic Qualities of Auditory Pitch , 1984, Perception.

[13]  Bruno L. Giordano.  Sound source perception in impact sounds , 2006 .

[14]  .. Wanderley.  Typology of Tactile Sounds and their Synthesis in Gesture-Driven Computer Music Performance , 2000 .

[15]  Lawrence E. Marks,et al.  Synesthesia: Strong and Weak , 2001 .

[16]  William W. Gaver.  How Do We Hear in the World?: Explorations in Ecological Acoustics , 1993 .

[17]  L E Marks,et al.  Perceptual and Linguistic Interactions in Speeded Classification: Tests of the Semantic Coding Hypothesis , 1999, Perception.

[18]  L E Marks,et al.  Perceiving similarity and comprehending metaphor. , 1988, Monographs of the Society for Research in Child Development.

[19]  Scott D. Lipscomb,et al.  PERCEIVED MATCH BETWEEN VISUAL PARAMETERS AND AUDITORY CORRELATES: AN EXPERIMENTAL MULTIMEDIA INVESTIGATION , 2004 .

[20]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[21]  V. Jousmäki,et al.  Parchment-skin illusion: sound-biased touch , 1998, Current Biology.

[22]  Susan J. Lederman,et al.  Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[23]  John G. Neuhoff,et al.  Dynamic frequency change influences loudness perception: a central, analytic process. , 1999, Journal of experimental psychology. Human perception and performance.

[24]  F. Rösler,et al.  Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli , 2002, Experimental Brain Research.

[25]  Brian Henson,et al.  Exploring Relationships between Touch Perception and Surface Physical Properties , 2009 .

[26]  Özgür Izmirli,et al.  Haptic and sound correlations: pitch, loudness and texture , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[27]  T Nakamura,et al.  The communication of dynamics between musicians and listeners through musical performance. , 1989, Perception & psychophysics.

[28]  S. Lederman.  Auditory Texture Perception , 1979, Perception.

[29]  M. Sile O'Modhrain,et al.  Cutaneous Grooves: Composing for the Sense of Touch , 2002 .

[30]  P. Walker,et al.  The Basis of Stroop Interference Involving the Multimodal Correlates of Auditory Pitch , 1986, Perception.

[31]  John J. Foxe,et al.  Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. , 2002, Journal of neurophysiology.

[32]  Z. Eitan,et al.  HOW MUSIC MOVES: Musical Parameters and Listeners' Images of Motion , 2006 .

[33]  D. Lewkowicz,et al.  The development of intersensory temporal perception: an epigenetic systems/limitations view. , 2000, Psychological bulletin.

[34]  R Walker,et al.  The effects of culture, environment, age, and musical training on choices of visual metaphors for sound , 1987, Perception & psychophysics.

[35]  David J. Lewkowicz,et al.  Cross-modal equivalence in early infancy: Auditory–visual intensity matching. , 1980 .

[36]  Zohar Eitan Nurit Tubul.  Musical Parameters and Children's Images of motion , 2010 .

[37]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[38]  J. M. Kittross.  The measurement of meaning , 1959 .

[39]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.