Attending to visual or auditory motion affects perception within and across modalities: an event‐related potential study

The present event‐related potential (ERP) study examined the role of dynamic features in multisensory binding. It was tested whether endogenous attention to the direction of motion affects processing of visual and auditory stimuli within and across modalities. Human participants perceived horizontally moving dot patterns and sounds that were presented either continuously (standards) or briefly interrupted (infrequent deviants). Their task was to detect deviants moving in a particular direction within a primary modality, but to detect all deviants irrespective of their motion direction within the secondary modality. Attending to the direction of visual motion resulted in a broad selection negativity (SN) starting at about 200 ms post‐stimulus onset, and attending to the direction of auditory motion resulted in a positive difference wave at 150 ms that was followed by a broad negativity starting at about 200 ms (unimodal effects). Moreover, dot patterns moving in a direction that was attended within audition were detected faster and more accurately than oppositely moving stimuli and elicited a cross‐modal SN wave. Corresponding cross‐modal behavioural and ERP results were obtained for sounds moving in a direction that was attended within vision. Unimodal and cross‐modal ERP attention effects partially differed in their scalp topography. The present study shows that dynamic features (direction of motion) may be used to link input across modalities and demonstrates for the first time that these multisensory interactions take place as early as about 200 ms after stimulus onset.

[1]  E. DeYoe,et al.  Graded effects of spatial and featural attention on human area MT and associated motion processing areas. , 1997, Journal of neurophysiology.

[2]  P. Cavanagh,et al.  Motion: the long and short of it. , 1989, Spatial vision.

[3]  R. Näätänen.  Processing negativity: an evoked-potential reflection of selective attention. , 1982, Psychological bulletin.

[4]  G. Sperling,et al.  The functional architecture of human visual motion perception , 1995, Vision Research.

[5]  Alan Kingstone,et al.  The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. , 2002, Brain research. Cognitive brain research.

[6]  Martin Eimer,et al.  Crossmodal links in spatial attention between vision, audition, and touch: evidence from event-related brain potentials , 2001, Neuropsychologia.

[7]  M. Posner.  Cognitive Science Program. Hierarchical Distributed Networks in the Neuropsychology of Selective Attention. , 1985 .

[8]  Insights from event-related potentials into the temporal and hierarchical organization of the ventral and dorsal streams of the visual system in selective attention. , 1999, Psychophysiology.

[9]  Jean Vroomen,et al.  Visual Motion Influences the Contingent Auditory Motion Aftereffect , 2003, Psychological science.

[10]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[11]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[12]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[13]  Alexander M. Harner,et al.  Task-dependent influences of attention on the activation of human primary visual cortex. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[14]  H Pratt,et al.  Ear advantage and attention: an ERP study of auditory cued attention , 2004, Hearing Research.

[15]  S. Hillyard,et al.  Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study. , 1999, Brain research. Cognitive brain research.

[16]  Helen J. Neville,et al.  Attention to central and peripheral visual space in a movement detection task: an event-related potential and behavioral study. I. Normal hearing adults , 1987, Brain Research.

[17]  Brigitte Röder,et al.  Unimodal and crossmodal effects of endogenous attention to visual and auditory motion , 2004, Cognitive, affective & behavioral neuroscience.

[18]  John Duncan,et al.  Restricted attentional capacity within but not between sensory modalities , 1997, Nature.

[19]  C. Frith,et al.  Processing of irrelevant visual motion during performance of an auditory attention task , 2001, Neuropsychologia.

[20]  S. Hillyard,et al.  Selective attention to multidimensional auditory stimuli. , 1983, Journal of experimental psychology. Human perception and performance.

[21]  H. Wilson,et al.  A psychophysically motivated model for two-dimensional motion perception , 1992, Visual Neuroscience.

[22]  Karl J. Friston,et al.  The functional anatomy of attention to visual motion. A functional MRI study. , 1998, Brain : a journal of neurology.

[23]  E. Yund,et al.  Central auditory processing IV. Ear dominance—Spatial and temporal complexity , 1983, Brain and Language.

[24]  B. Bauer,et al.  Phasor analysis of some stereophonic phenomena , 1962 .

[25]  W. Ziegler,et al.  Contrasting spatial hearing deficits in hemianopia and spatial neglect. , 1999, Neuroreport.

[26]  G H Recanzone,et al.  Effects of attention on MT and MST neuronal activity during pursuit initiation. , 2000, Journal of neurophysiology.

[27]  M. Farah,et al.  Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal? , 1989, Neuropsychologia.

[28]  S. Hillyard,et al.  Selective attention to the color and direction of moving stimuli: Electrophysiological correlates of hierarchical feature selection , 1996, Perception & psychophysics.

[29]  T A Mondor,et al.  On the relation between auditory spatial attention and auditory perceptual asymmetries , 1992, Perception & psychophysics.

[30]  J. Driver,et al.  Audiovisual links in endogenous covert spatial attention. , 1996, Journal of experimental psychology. Human perception and performance.

[31]  M. Rizzo,et al.  The Effect of Focal Cerebral Lesions on Intramodal and Cross-modal Orienting of Attention , 1989 .

[32]  Benjamin B. Bauer.  Phasor Analysis of the Stereophonic Phenomena , 1961 .

[33]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[34]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[35]  O J Braddick,et al.  Low-level and high-level processes in apparent motion. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[36]  A. Treisman,et al.  Voluntary Attention Modulates fMRI Activity in Human MT–MST , 1997, Neuron.

[37]  D W Grantham,et al.  Detection and discrimination of simulated motion of auditory targets in the horizontal plane. , 1986, The Journal of the Acoustical Society of America.

[38]  R. Turner,et al.  Form and motion coherence activate independent, but not dorsal/ventral segregated, networks in the human brain , 2000, Current Biology.

[39]  S. Hillyard,et al.  Electrical Signs of Selective Attention in the Human Brain , 1973, Science.

[40]  R Sekuler,et al.  Mental set alters visibility of moving targets , 1977, Science.

[41]  S. Treue,et al.  Attentional Modulation Strength in Cortical Area MT Depends on Stimulus Contrast , 2002, Neuron.

[42]  C. C. Wood,et al.  Scalp distributions of event-related potentials: an ambiguity associated with analysis of variance models. , 1985, Electroencephalography and clinical neurophysiology.

[43]  S. Mateeff,et al.  Dynamic Visual Capture: Apparent Auditory Motion Induced by a Moving Visual Target , 1985, Perception.

[44]  Carol L Colby,et al.  Auditory and visual attention modulate motion processing in area MT+. , 2002, Brain research. Cognitive brain research.

[45]  Helen J. Neville,et al.  Attention to central and peripheral visual space in a movement detection task: an event-related potential and behavioral study. II. Congenitally deaf adults , 1987, Brain Research.

[46]  Risto N t nen.  Processing negativity: An evoked-potential reflection. , 1982 .

[47]  B. Röder,et al.  Attention to motion enhances processing of both visual and auditory stimuli: an event-related potential study. , 2004, Brain research. Cognitive brain research.

[48]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[49]  G. Mangun.  Neural mechanisms of visual selective attention. , 1995, Psychophysiology.

[50]  Christopher D. Wickens,et al.  The Structure of Attentional Resources , 1980 .

[51]  Steven A. Hillyard,et al.  Neural Substrates of Perceptual Enhancement by Cross-Modal Spatial Attention , 2003, Journal of Cognitive Neuroscience.