Multisensory Interactions Elicited by Audiovisual Stimuli Presented Peripherally in a Visual Attention Task: A Behavioral and Event-Related Potential Study in Humans

We applied behavioral and event-related potential measurements to study human multisensory interactions induced by audiovisual (AV) stimuli presented peripherally in a visual attention task in which an irrelevant auditory stimulus occasionally accompanied the visual stimulus. A stream of visual, auditory, and AV stimuli was randomly presented to the left or right side of the subjects; subjects covertly attended to the visual stimuli on either the left or right side and promptly responded to visual targets on that side. Behavioral results showed that responses to AV stimuli were faster and more accurate than those to visual stimuli only. Three event-related potential components related to AV interactions were identified: (1) over the right temporal area, approximately 200 to 220 milliseconds; (2) over the centromedial area, approximately 290 to 310 milliseconds; and (3) over the left and right ventral temporal area, approximately 290 to 310 milliseconds. We found that these interaction effects occurred slightly later than those reported in previously published AV interaction studies in which AV stimuli were presented centrally. Our results suggest that the retinotopic location of stimuli affects AV interactions occurring at later stages of cognitive processing in response to a visual attention task.

[1]  P. Goolkasian,et al.  Retinal location and its effect on the spatial distribution of visual attention. , 1999, The American journal of psychology.

[2]  M. Carrasco,et al.  Attention alters appearance , 2004, Nature Neuroscience.

[3]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[4]  N. Bruneau,et al.  Cross-modal processing of auditory–visual stimuli in a no-task paradigm: A topographic event-related potential study , 2008, Clinical Neurophysiology.

[5]  L M Ward,et al.  Involuntary Listening Aids Seeing: Evidence From Human Electrophysiology , 2000, Psychological science.

[6]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[7]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[8]  J. Pernier,et al.  Early auditory-visual interactions in human cortex during nonredundant target identification. , 2002, Brain research. Cognitive brain research.

[9]  Mark T. Wallace,et al.  Development and neural basis of multisensory integration. , 1994 .

[10]  Marty G. Woldorff,et al.  Selective Attention and Multisensory Integration: Multiple Phases of Effects on the Evoked Brain Activity , 2005, Journal of Cognitive Neuroscience.

[11]  Geraint Rees,et al.  Sound alters activity in human V1 in association with illusory visual perception , 2006, NeuroImage.

[12]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[13]  Brigitte Röder,et al.  Multisensory processing in the redundant-target effect: A behavioral and event-related potential study , 2005, Perception & psychophysics.

[14]  R. Anderson,et al.  Aliasing in peripheral vision for counterphase gratings. , 1996, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  Simon Rigoulot,et al.  Peripherally Presented Emotional Scenes: A Spatiotemporal Analysis of Early ERP Responses , 2008, Brain Topography.

[16]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[17]  S. Hillyard,et al.  Involuntary orienting to sound improves visual perception , 2000, Nature.

[18]  E. Schröger,et al.  Speeded responses to audiovisual signal changes result from bimodal integration. , 1998, Psychophysiology.

[19]  Steven A. Hillyard,et al.  Effects of Spatial Congruity on Audio-Visual Multimodal Integration , 2005, Journal of Cognitive Neuroscience.

[20]  Martin Eimer,et al.  An ERP study of sustained spatial attention to stimulus eccentricity , 2000, Biological Psychology.

[21]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[22]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[23]  Todd C. Handy,et al.  Attention and Sensory Gain Control: A Peripheral Visual Process? , 2005, Journal of Cognitive Neuroscience.

[24]  S A Hillyard,et al.  An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. , 2002, Brain research. Cognitive brain research.