On the relative contributions of multisensory integration and crossmodal exogenous spatial attention to multisensory response enhancement.

Two processes that can give rise to multisensory response enhancement (MRE) are multisensory integration (MSI) and crossmodal exogenous spatial attention. It is, however, currently unclear what the relative contribution of each of these is to MRE. We investigated this issue using two tasks that are generally assumed to measure MSI (a redundant target effect task) and crossmodal exogenous spatial attention (a spatial cueing task). One block of trials consisted of unimodal auditory and visual targets designed to provide a unimodal baseline. In two other blocks of trials, the participants were presented with spatially and temporally aligned and misaligned audiovisual (AV) targets (0, 50, 100, and 200ms SOA). In the integration block, the participants were instructed to respond to the onset of the first target stimulus that they detected (A or V). The instruction for the cueing block was to respond only to the onset of the visual targets. The targets could appear at one of three locations: left, center, and right. The participants were instructed to respond only to lateral targets. The results indicated that MRE was caused by MSI at 0ms SOA. At 50ms SOA, both crossmodal exogenous spatial attention and MSI contributed to the observed MRE, whereas the MRE observed at the 100 and 200ms SOAs was attributable to crossmodal exogenous spatial attention, alerting, and temporal preparation. These results therefore suggest that there may be a temporal window in which both MSI and exogenous crossmodal spatial attention can contribute to multisensory response enhancement.

[1]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[2]  Rolf Ulrich,et al.  Testing the race model inequality: An algorithm and computer programs , 2007, Behavior research methods.

[3]  J. Vroomen,et al.  Perception of intersensory synchrony: A tutorial review , 2010, Attention, perception & psychophysics.

[4]  Paul J. Laurienti,et al.  Modality-specific selective attention attenuates multisensory integration , 2007, Experimental Brain Research.

[5]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[6]  L. Busse,et al.  The spread of attention across modalities and space in a multisensory object. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[7]  L M Ward,et al.  Multisensory integration and crossmodal attention effects in the human brain. , 2001, Science.

[8]  Hans Colonius,et al.  Modeling Multisensory Processes in Saccadic Responses: Time-Window-of-Integration Model , 2009 .

[9]  Jennifer L. Mozolic,et al.  Suppression of multisensory integration by modality-specific attention in aging , 2009, Neuroreport.

[10]  J. Driver,et al.  Audiovisual links in exogenous covert spatial orienting , 1997, Perception & psychophysics.

[11]  Steven A. Hillyard,et al.  Neural Substrates of Perceptual Enhancement by Cross-Modal Spatial Attention , 2003, Journal of Cognitive Neuroscience.

[12]  Viola S. Störmer,et al.  Cross-modal cueing of attention alters appearance and early cortical processing of visual stimuli , 2009, Proceedings of the National Academy of Sciences.

[13]  C. Frith,et al.  Multisensory integration and crossmodal attention effects in the human brain - Response , 2001 .

[14]  P. Bertelson,et al.  The ventriloquist effect does not depend on the direction of automatic visual attention , 2001, Perception & psychophysics.

[15]  B. Stein,et al.  Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  A. Diederich,et al.  Multisensory Interaction in Saccadic Reaction Time: A Time-Window-of-Integration Model , 2004, Journal of Cognitive Neuroscience.

[17]  D. Raab.  Statistical facilitation of simple reaction times. , 1962, Transactions of the New York Academy of Sciences.

[18]  L M Ward,et al.  Involuntary Listening Aids Seeing: Evidence From Human Electrophysiology , 2000, Psychological science.

[19]  Hans Colonius,et al.  Crossmodal interaction in saccadic reaction time: separating multisensory from warning effects in the time window of integration model , 2008, Experimental Brain Research.

[20]  B. Stein,et al.  Multisensory integration. Neural and behavioral solutions for dealing with stimuli from different sensory modalities. , 1990, Annals of the New York Academy of Sciences.

[21]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[22]  Bettina Forster,et al.  An ERP Investigation on Visuotactile Interactions in Peripersonal and Extrapersonal Space: Evidence for the Spatial Rule , 2009, Journal of Cognitive Neuroscience.

[23]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[24]  C. Spence,et al.  Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli , 2010, Attention, perception & psychophysics.

[25]  L. M. Ward,et al.  Spatial relevance determines facilitatory and inhibitory effects of auditory covert spatial orienting , 1999 .

[26]  A. Serino,et al.  Dynamic Sounds Capture the Boundaries of Peripersonal Space Representation in Humans , 2012, PloS one.

[27]  C. Spence,et al.  Cross-modal links in spatial attention. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[28]  Pascal Mamassian,et al.  Noise and Correlations in Parallel Perceptual Decision Making , 2011 .

[29]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[30]  B. Stein,et al.  The Merging of the Senses , 1993 .

[31]  S. van der Stigchel,et al.  Exogenous orienting of crossmodal attention in 3-D space: Support for a depth-aware crossmodal attentional system , 2013, Psychonomic Bulletin & Review.

[32]  John J. Foxe,et al.  Neuro-Oscillatory Phase Alignment Drives Speeded Multisensory Response Times: An Electro-Corticographic Investigation , 2015, The Journal of Neuroscience.

[33]  M. Keeling,et al.  Involuntary orienting to sound improves visual perception , 2022 .

[34]  Murray Mm,et al.  Modeling Multisensory Processes in Saccadic Responses: Time-Window-of-Integration Model -- The Neural Bases of Multisensory Processes , 2012 .

[35]  Charles Spence,et al.  Capturing spatial attention with multisensory cues: A review , 2009, Hearing Research.

[36]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[37]  C. Spence,et al.  Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism , 2000, Neuroreport.

[38]  Sander A. Los,et al.  The effective time course of preparation , 2008, Cognitive Psychology.

[39]  D. Raab.  DIVISION OF PSYCHOLOGY: STATISTICAL FACILITATION OF SIMPLE REACTION TIMES* , 1962 .

[40]  C. Spence,et al.  Multisensory interactions in the depth plane in front and rear space: A review , 2015, Neuropsychologia.

[41]  S. van der Stigchel,et al.  Exogenous spatial attention decreases audiovisual integration , 2014, Attention, Perception, & Psychophysics.

[42]  Hans Colonius,et al.  Focused attention vs. crossmodal signals paradigm: deriving predictions from the time-window-of-integration model , 2012, Front. Integr. Neurosci..

[43]  C. Spence,et al.  Multisensory prior entry. , 2001, Journal of experimental psychology. General.

[44]  M. McCourt,et al.  The roles of physical and physiological simultaneity in audiovisual multisensory facilitation , 2013, i-Perception.

[45]  J. Navarra,et al.  Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task , 2004, Cognition.

[46]  B. Stein,et al.  Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. , 1986, Journal of neurophysiology.

[47]  Brigitte Röder,et al.  The redundant target effect is affected by modality switch costs , 2004, Psychonomic bulletin & review.

[48]  Aaron R. Nidiffer,et al.  Identifying and Quantifying Multisensory Integration: A Tutorial Review , 2014, Brain Topography.

[49]  C. Spence,et al.  Prior-entry: A review , 2010, Consciousness and Cognition.

[50]  M. Wallace,et al.  Enhanced multisensory integration in older adults , 2006, Neurobiology of Aging.

[51]  John J. Foxe,et al.  Audio-visual multisensory integration in superior parietal lobule revealed by human intracranial recordings. , 2006, Journal of neurophysiology.

[52]  Erik Van der Burg,et al.  Sound speeds vision through preparation, not integration. , 2013, Journal of experimental psychology. Human perception and performance.

[53]  C. Spence.  Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule , 2013, Annals of the New York Academy of Sciences.

[54]  Brigitte Röder,et al.  Multisensory processing in the redundant-target effect: A behavioral and event-related potential study , 2005, Perception & psychophysics.

[55]  R. Rafal,et al.  Competition between endogenous and exogenous orienting of visual attention. , 2005, Journal of experimental psychology. General.

[56]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.

[57]  Jeff Miller,et al.  Systematic biases and Type I error accumulation in tests of the race model inequality , 2007, Behavior research methods.

[58]  Jeff Miller,et al.  Timecourse of coactivation in bimodal divided attention , 1986, Perception & psychophysics.

[59]  Zachary P. Barnett,et al.  Interactions between the spatial and temporal stimulus factors that influence multisensory integration in human performance , 2012, Experimental Brain Research.

[60]  Ning Liu,et al.  A novel, variable angle guide grid for neuronal activity studies , 2011, Front. Integr. Neurosci..

[61]  P Bertelson,et al.  Directing spatial attention towards the illusory location of a ventriloquized sound. , 2001, Acta psychologica.

[62]  L. M. Ward,et al.  On asymmetries in cross-modal spatial attention orienting , 2000, Perception & psychophysics.

[63]  S. Hillyard,et al.  Neural basis of auditory-induced shifts in visual time-order perception , 2005, Nature Neuroscience.

[64]  C. Spence.  Crossmodal spatial attention , 2010, Annals of the New York Academy of Sciences.

[65]  M A Meredith,et al.  Multisensory Integration , 1990 .