Changing pitch induced visual motion illusion

We often associate moving objects and changing pitch, e.g., falling stones with descending, and launching rockets with ascending pitch, even when these sounds do not happen in the real-world. The reason for this is unknown. Here we report an illusion in which auditory stimuli with no apparent spatial and motion information [[1–3]] alter human visual motion perception. Subjects made a two alternative forced choice (upward (Vup) or downward (Vdown) visual motion perception) while presented with two superimposed, oppositely moving gratings (experiment 1), accompanied by either an ascending or a descending pitch of pure tone, or broad-band noise (Figure 1A). Gratings with ambiguous motion accompanied by ascending pitch were more likely to be perceived as an upward motion, those accompanied by descending pitch as a downward motion, whereas noise caused no directional bias.

[1]  O. C. Trimble LOCALIZATION OF SOUND IN THE ANTERIOR‐POSTERIOR AND VERTICAL DIMENSIONS OF “AUDITORY” SPACE1 , 1934 .

[2]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[3]  S. Wuerger,et al.  Cross-modal integration of auditory and visual motion signals , 2001, Neuroreport.

[4]  Mortimer Mishkin,et al.  Mapping the primate visual system with [2-14C]deoxyglucose. , 1982, Science.

[5]  R. Butler The Unfolding of an Auditory Illusion , 2015, Perspectives in biology and medicine.

[6]  J. C. Middlebrooks Narrow-band sound localization related to external ear acoustics. , 1992, The Journal of the Acoustical Society of America.

[7]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[8]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[9]  Y. Sugita,et al.  Audiovisual perception: Implicit estimation of sound-arrival time , 2003, Nature.

[10]  Alan Kingstone,et al.  The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. , 2002, Brain research. Cognitive brain research.

[11]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[12]  I H Bernstein,et al.  Effects of some variations in auditory input upon visual choice reaction time. , 1971, Journal of experimental psychology.

[13]  R. Sekuler,et al.  Cues reduce direction uncertainty and enhance motion detection , 1981, Perception & psychophysics.

[14]  R. Andersen,et al.  Spatially tuned auditory responses in area LIP of macaques performing delayed memory saccades to acoustic targets. , 1996, Journal of neurophysiology.

[15]  Daniel J Tollin,et al.  Spectral cues explain illusory elevation effects with stereo sounds in cats. , 2003, Journal of neurophysiology.

[16]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[17]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[18]  M. Mishkin,et al.  Functional Mapping of the Primate Auditory System , 2003, Science.

[19]  Jean Vroomen,et al.  Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity , 2004, Neuroscience Letters.

[20]  R. Butler,et al.  Localization of tonal stimuli in the vertical plane. , 1968, The Journal of the Acoustical Society of America.

[21]  Yôiti Suzuki,et al.  Implicit estimation of sound-arrival time , 2003, Nature.

[22]  J. H. Neely Semantic priming and retrieval from lexical memory: Roles of inhibitionless spreading activation and limited-capacity attention. , 1977 .

[23]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[24]  S. Shimojo,et al.  Sound alters visual evoked potentials in humans , 2001, Neuroreport.

[25]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[26]  Lawrence E. Marks,et al.  Visual-auditory interaction in speeded classification: Role of stimulus difference , 1995, Perception & psychophysics.

[27]  Tracey D. Berger,et al.  Flicker flutter: is an illusory event as good as the real thing? , 2003, Journal of vision.

[28]  Shigeto Furukawa,et al.  Auditory cortical responses in the cat to sounds that produce spatial illusions , 1999, Nature.

[29]  A. Gorea,et al.  Perceptual bistability with counterphase gratings , 1984, Vision Research.

[30]  L E Marks,et al.  Dimensional interactions in language processing: investigating directions and levels of crosstalk. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[31]  M. Hallett,et al.  Neural correlates of cross-modal binding , 2003, Nature Neuroscience.

[32]  C. C. Pratt The spatial character of high and low tones. , 1930 .

[33]  Ellen Winner,et al.  "Metaphorical" Mapping in Human Infants , 1981 .