Audio-visual integration in temporal perception.

In situations of audio-visual interaction, research has generally found that audition prevails over vision in temporal perception, while vision is dominant over audition for spatial perception. Modality appropriateness to a given task generally determines the direction of this inter-modality effect. However, we found a reverse effect in some situations where a change in the frequency of visual stimuli was associated with a perceived change in the frequency of auditory stimuli. In our experiment, 12 participants were asked to judge the change in the frequency of visual and auditory stimuli using a visual flicker and auditory flutter stimuli. In some conditions either the auditory or the visual information was ambiguous. In addition to confirming the expected finding that a change in the frequency of the auditory stimuli induced a perceived change in the frequency of the visual stimuli, we found a new phenomenon. When ambiguous auditory temporal cues were presented, the change in the frequency of the visual stimuli was associated with a perceived change in the frequency of the auditory stimuli. This suggests that cross-modal asymmetry effects are influenced by the reliability of visual and auditory information as well as modality appropriateness.

[1]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[2]  Mowbray Gh,et al.  On discriminating the rate of visual flicker and auditory flutter. , 1959 .

[3]  Robert B. Welch,et al.  Contributions of audition and vision to temporal rate perception , 1986, Perception & psychophysics.

[4]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[5]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[6]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[7]  D. Wolpert,et al.  When Feeling Is More Important Than Seeing in Sensorimotor Adaptation , 2002, Current Biology.

[8]  S. Wuerger,et al.  Cross-modal integration of auditory and visual motion signals , 2001, Neuroreport.

[9]  G. Aschersleben,et al.  Automatic visual bias of perceived auditory location , 1998 .

[10]  Alexandre Pouget,et al.  A computational perspective on the neural basis of multisensory spatial representations , 2002, Nature Reviews Neuroscience.

[11]  R. Jacobs,et al.  Optimal integration of texture and motion cues to depth , 1999, Vision Research.

[12]  M. Mon-Williams,et al.  The contribution of vision and proprioception to judgements of finger proximity , 1998, Experimental Brain Research.

[13]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[14]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[15]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[16]  M. Mon-Williams,et al.  Synaesthesia in the normal limb , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[17]  S. Shimojo,et al.  Sound alters visual evoked potentials in humans , 2001, Neuroreport.

[18]  T SHIPLEY,et al.  Auditory Flutter-Driving of Visual Flicker , 1964, Science.

[19]  S. Shimojo,et al.  Sensory modalities are not separate modalities: plasticity and interactions , 2001, Current Opinion in Neurobiology.

[20]  H Spekreijse,et al.  Auditory—Visual Interactions and the Correspondence between Perceived Auditory Space and Perceived Visual Space , 1977, Perception.

[21]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[22]  R. J. van Beers,et al.  Integration of proprioceptive and visual position-information: An experimentally supported model. , 1999, Journal of neurophysiology.

[23]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[24]  W R Thurlow,et al.  Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. , 1973, Perceptual and motor skills.

[25]  J. J. Denier van der Gon,et al.  Integration of proprioceptive and visual position-information , 1999 .