Feeling what you hear: auditory signals can modulate tactile tap perception

We tested whether auditory sequences of beeps can modulate the tactile perception of sequences of taps (two to four taps per sequence) delivered to the index fingertip. In the first experiment, the auditory and tactile sequences were presented simultaneously. The number of beeps delivered in the auditory sequence were either the same as, less than, or more than the number of taps of the simultaneously presented tactile sequence. Though task-irrelevant (subjects were instructed to focus on the tactile stimuli), the auditory stimuli systematically modulated subjects’ tactile perception; in other words subjects’ responses depended significantly on the number of delivered beeps. Such modulation only occurred when the auditory and tactile stimuli were similar enough. In the second experiment, we tested whether the automatic auditory-tactile integration depends on simultaneity or whether a bias can be evoked when the auditory and tactile sequence are presented in temporal asynchrony. Audition significantly modulated tactile perception when the stimuli were presented simultaneously but this effect gradually disappeared when a temporal asynchrony was introduced between auditory and tactile stimuli. These results show that when provided with auditory and tactile sensory signals that are likely to be generated by the same stimulus, the central nervous system (CNS) tends to automatically integrate these signals.

[1]  M HERSHENSON,et al.  Reaction time as a measure of intersensory facilitation. , 1962, Journal of experimental psychology.

[2]  L. Morrell,et al.  Temporal characteristics of sensory interaction in choice reaction times. , 1968, Journal of experimental psychology.

[3]  I H Bernstein,et al.  Effects of an auditory signal on visual reaction time. , 1969, Journal of experimental psychology.

[4]  R S Nickerson,et al.  Intersensory facilitation of reaction time: energy summation or preparation enhancement? , 1973, Psychological review.

[5]  J. Hyvärinen,et al.  Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys. , 1974, Brain : a journal of neurology.

[6]  R B Welch,et al.  Effect of Degree of Separation of Visual-Auditory Stimulus and Eye Position upon Spatial Interaction of Vision and Audition , 1976, Perceptual and motor skills.

[7]  R. Desimone,et al.  Visual areas in the temporal cortex of the macaque , 1979, Brain Research.

[8]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[9]  R. Schmidt,et al.  On the nature of intersensory facilitation of reaction time , 1983, Perception & psychophysics.

[10]  Keiji Tanaka,et al.  Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. , 1988, Journal of neurophysiology.

[11]  M. Abbott,et al.  What You See Is What You Are , 1993 .

[12]  G Aschersleben,et al.  Synchronizing actions with events: The role of sensory information , 1995, Perception & psychophysics.

[13]  Mandayam A. Srinivasan,et al.  The Effect of Auditory Cues on the Haptic Perception of Stiffness in Virtual Environments , 1997, Dynamic Systems and Control.

[14]  V. Jousmäki,et al.  Parchment-skin illusion: sound-biased touch , 1998, Current Biology.

[15]  M. Srinivasan,et al.  Visual, Haptic, and Bimodal Perception of Size and Stiffness in Virtual Environments , 1999, Dynamic Systems and Control.

[16]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[17]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[18]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[19]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[20]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[21]  C. Spence,et al.  Audiotactile interactions in roughness perception , 2002, Experimental Brain Research.

[22]  John J. Foxe,et al.  Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. , 2002, Journal of neurophysiology.

[23]  S. Shimojo,et al.  Visual illusion induced by sound. , 2002, Brain research. Cognitive brain research.

[24]  C. Lammertmann,et al.  Magnetoencephalographic Correlates of Audiotactile Interaction , 2002, NeuroImage.

[25]  Knut Drewing,et al.  Temporal coordination of simple movements , 2002 .

[26]  S. Gepshtein,et al.  Viewing Geometry Determines How Vision and Haptics Combine in Size Perception , 2003, Current Biology.

[27]  Riitta Hari,et al.  Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction , 2003, NeuroImage.

[28]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[29]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[30]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[31]  J. Hyvärinen,et al.  Functional properties of neurons in the temporo-parietal association cortex of awake monkey , 2004, Experimental Brain Research.

[32]  Brigitte Röder,et al.  Hearing Cheats Touch, but Less in Congenitally Blind Than in Sighted Individuals , 2004, Psychological science.