Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities.

This study investigated multisensory interactions in the perception of auditory and visual motion. When auditory and visual apparent motion streams are presented concurrently in opposite directions, participants often fail to discriminate the direction of motion of the auditory stream, whereas perception of the visual stream is unaffected by the direction of auditory motion (Experiment 1). This asymmetry persists even when the perceived quality of apparent motion is equated for the 2 modalities (Experiment 2). Subsequently, it was found that this visual modulation of auditory motion is caused by an illusory reversal in the perceived direction of sounds (Experiment 3). This "dynamic capture" effect occurs over and above ventriloquism among static events (Experiments 4 and 5), and it generalizes to continuous motion displays (Experiment 6). These data are discussed in light of related multisensory phenomena and their support for a "modality appropriateness" interpretation of multisensory integration in motion perception.

[1]  Robert B. Welch,et al.  Contributions of audition and vision to temporal rate perception , 1986, Perception & psychophysics.

[2]  O. Hikosaka,et al.  Voluntary and Stimulus-Induced Attention Detected as Motion Sensation , 1993, Perception.

[3]  Richard S. J. Frackowiak,et al.  Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. , 1993, Cerebral cortex.

[4]  R. Klein.  Attention and visual dominance: a chronometric analysis. , 1977, Journal of experimental psychology. Human perception and performance.

[5]  H. E. Burtt.  Auditory illusions of movement - A preliminary study , 1917 .

[6]  R. Welch.  Chapter 15 Meaning, attention, and the “unity assumption” in the intersensory bias of spatial and temporal perceptions , 1999 .

[7]  S Lakatos,et al.  The Influence of Visual Cues on the Localisation of Circular Auditory Motion , 1995, Perception.

[8]  B. Gibson,et al.  Attraction, Distraction and Action: Multiple Perspectives on Attentional Capture. Advances in Psychology , 2001 .

[9]  S. Soto-Faraco.  Multisensory integration of dynamic information , 2004 .

[10]  S. Charles,et al.  10 - Crossmodal Attentional Capture: A Controversy Resolved? , 2001 .

[11]  Victor Urbantschitsch,et al.  Ueber den Einfluss einer Sinneserregung auf die übrigen Sinnesempfindungen , 1888, Archiv für die gesamte Physiologie des Menschen und der Tiere.

[12]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[13]  Anya Hurlbert,et al.  Interactions between visual and auditory movement perception in a direction discrimination task , 2000 .

[14]  D McAlpine,et al.  Spatial receptive fields of inferior colliculus neurons to auditory apparent motion in free field. , 2001, Journal of neurophysiology.

[15]  Jean Vroomen,et al.  Visual Motion Influences the Contingent Auditory Motion Aftereffect , 2003, Psychological science.

[16]  Steven Connor,et al.  Dumbstruck: A Cultural History of Ventriloquism , 2000 .

[17]  D. H. Warren,et al.  Sensory conflict in judgments of spatial direction , 1969 .

[18]  A. Rees,et al.  Evidence for a sound movement area in the human cerebral cortex , 1996, Nature.

[19]  D. Freides,et al.  Human information processing and sensory modality: cross-modal functions, information complexity, memory, and deficit. , 1974, Psychological bulletin.

[20]  I. Howard,et al.  Human Spatial Orientation , 1966 .

[21]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[22]  B. Stein,et al.  The Merging of the Senses , 1993 .

[23]  R. S. J. Frackowiak,et al.  Human cortical areas selectively activated by apparent sound movement , 1994, Current Biology.

[24]  P. Bertelson,et al.  The psychology of multimodal perception , 2004 .

[25]  Alan Kingstone,et al.  The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. , 2002, Brain research. Cognitive brain research.

[26]  Robert B. Welch,et al.  The “ventriloquist effect”: Visual dominance or response bias? , 1975 .

[27]  G. M. Gilbert,et al.  Inter-Sensory Facilitation and Inhibition , 1941 .

[28]  Joel L. Davis,et al.  Large-Scale Neuronal Theories of the Brain , 1994 .

[29]  D C Donderi,et al.  The effect of sound on visual apparent movement. , 1983, The American journal of psychology.

[30]  P. Bertelson.  Chapter 14 Ventriloquism: A case of crossmodal perceptual grouping , 1999 .

[31]  N. Kitajima,et al.  Dynamic Capture of Sound Motion by Light Stimuli Moving in Three-Dimensional Space , 1999, Perceptual and motor skills.

[32]  P. A. Kolers.  THE ILLUSION OF MOVEMENT. , 1964, Scientific American.

[33]  S M Anstis.  Hearing with the Hands , 1973, Perception.

[34]  P. A. Kolers,et al.  Sensory specificity of apparent motion. , 1981, Journal of experimental psychology. Human perception and performance.

[35]  T. Sejnowski,et al.  A critique of pure vision , 1993 .

[36]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[37]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[38]  G. C. Zapparoli,et al.  The apparent movement between visual and acoustic stimulus and the problem of intermodal relations. , 1969, Acta psychologica.

[39]  S. Wuerger,et al.  Cross-modal integration of auditory and visual motion signals , 2001, Neuroreport.

[40]  G. Aschersleben,et al.  Automatic visual bias of perceived auditory location , 1998 .

[41]  H. Ohmura.  Intersensory influences on the perception of apparent movement , 1987 .

[42]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[43]  Paul Bertelson,et al.  Ventriloquism, sensory interaction, and response bias: Remarks on the paper by Choe, Welch, Gilford, and Juola , 1976 .

[44]  D. Massaro.  Speech Perception By Ear and Eye: A Paradigm for Psychological Inquiry , 1989 .

[45]  W R Thurlow,et al.  Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. , 1973, Perceptual and motor skills.

[46]  T. Bachmann,et al.  Cognitive contributions to the perception of spatial and temporal events (Advances in Psychology, Vol. 129) , 1999 .

[47]  C. Spence,et al.  Tactile “capture” of audition , 2002, Perception & psychophysics.

[48]  R. Shepard,et al.  Constraints common to apparent motion in visual, tactile, and auditory space. , 1997, Journal of Experimental Psychology: Human Perception and Performance.

[49]  C. Spence,et al.  Multisensory contributions to the perception of motion , 2003, Neuropsychologia.

[50]  Emmanuel Dupoux,et al.  Expe: An expandable programming language for on-line psychological experiments , 1997 .

[51]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[52]  D. C. Howell.  Statistical methods for psychology, 3rd ed. , 1992 .

[53]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.