Dual mechanisms for the cross-sensory spread of attention: how much do learned associations matter?

The spread of attention-related processing across anatomically separated cortical regions plays an important role in the binding of an object's features, both within and across sensory modalities. We presented multiple exemplars of semantically congruent multisensory objects (e.g., dogs with barks) and semantically incongruent multisensory objects (e.g., guitars with barks) while recording high-density event-related potentials and tested whether highly learned associations among the multisensory features of well-known objects modulated the spread of attention from an attended visual stimulus to its paired, task-irrelevant sound. Our findings distinguish dual mechanisms for the cross-sensory spread of attention: 1) a stimulus-driven spread of attention that occurs whenever a task-irrelevant sound is simultaneously presented with an attended visual stimulus, independent of highly learned associations, and 2) a representation-driven spread of attention that occurs in response to a task-irrelevant sound that is semantically congruent with a visual target and is therefore dependent on highly learned associations. The first mechanism is thought to reflect bottom-up feature binding, whereas the second mechanism is thought to reflect the top-down activation of a stored object representation that includes the well-known object's multisensory features. When a semantically congruent, task-irrelevant sound is simultaneously presented with a well-known visual target, the combined spread of attention through both mechanisms appears additive.

[1]  John J. Foxe,et al.  Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. , 2004, Cerebral cortex.

[2]  R Kakigi,et al.  [Event-related brain potentials]. , 1997, Nihon rinsho. Japanese journal of clinical medicine.

[3]  E. Schröger,et al.  ERP effects of intermodal attention and cross-modal links in spatial attention. , 1998, Psychophysiology.

[4]  J. Stroop.  Studies of interference in serial verbal reactions. , 1992 .

[5]  C. C. Wood,et al.  Scalp distributions of event-related potentials: an ambiguity associated with analysis of variance models. , 1985, Electroencephalography and clinical neurophysiology.

[6]  Durk Talsma,et al.  Nonspatial intermodal selective attention is mediated by sensory brain areas: Evidence from event-related potentials , 2001 .

[7]  D. Guthrie,et al.  Significance testing of difference potentials. , 1991, Psychophysiology.

[8]  K. Crowley,et al.  A review of the evidence for P2 being an independent component process: age, sleep and modality , 2004, Clinical Neurophysiology.

[9]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[10]  L. Busse,et al.  The spread of attention across modalities and space in a multisensory object. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[11]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[12]  Nancy Kanwisher,et al.  fMRI evidence for objects as the units of attentional selection , 1999, Nature.

[13]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[14]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[15]  M. Keeling,et al.  Involuntary orienting to sound improves visual perception , 2022 .

[16]  Erik Blaser,et al.  Tracking an object through feature space , 2000, Nature.

[17]  John J. Foxe,et al.  Jumping the gun: is effective preparation contingent upon anticipatory activation in task-relevant neural circuitry? , 2006, Cerebral cortex.

[18]  John J. Foxe,et al.  Object‐based attention is multisensory: co‐activation of an object's representations in ignored sensory modalities , 2007, The European journal of neuroscience.

[19]  John J. Foxe,et al.  Task switching: a high-density electrical mapping study , 2003, NeuroImage.

[20]  Randolph D. Easton,et al.  Visual–Auditory Events: Cross-Modal Perceptual Priming and Recognition Memory , 2001, Consciousness and Cognition.

[21]  R. Desimone,et al.  Responses of Neurons in Inferior Temporal Cortex during Memory- Guided Visual Search , 1998 .

[22]  Steven A. Hillyard,et al.  Objects Are Highlighted by Spatial Attention , 2006 .

[23]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[24]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[25]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[26]  R. Näätänen,et al.  Early selective-attention effect on evoked potential reinterpreted. , 1978, Acta psychologica.

[27]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[28]  L. Robertson.  Binding, spatial attention and perceptual awareness , 2003, Nature Reviews Neuroscience.

[29]  S. Hillyard,et al.  Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study. , 1999, Brain research. Cognitive brain research.

[30]  David Melcher,et al.  Subthreshold features of visual objects: Unseen but not unbound , 2006, Vision Research.

[31]  S. Hillyard,et al.  Endogenous brain potentials associated with selective auditory attention. , 1980, Electroencephalography and clinical neurophysiology.

[32]  R. Rafal,et al.  Shifting visual attention between objects and locations: evidence from normal and parietal lesion subjects. , 1994, Journal of experimental psychology. General.

[33]  J-M Hopf,et al.  Dynamics of feature binding during object-selective attention , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Antígona Martínez,et al.  Spatial attention facilitates selection of illusory objects: Evidence from event-related brain potentials , 2007, Brain Research.

[35]  G. Boynton,et al.  Global feature-based attention for motion and color , 2003, Vision Research.

[36]  J. Duncan.  EPS Mid-Career Award 2004: Brain mechanisms of attention , 2006, Quarterly journal of experimental psychology.

[37]  Steven A. Hillyard,et al.  Neural Substrates of Perceptual Enhancement by Cross-Modal Spatial Attention , 2003, Journal of Cognitive Neuroscience.

[38]  John Duncan,et al.  A neural basis for visual search in inferior temporal cortex , 1993, Nature.

[39]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[40]  John J. Foxe,et al.  Don't think of a white bear: An fMRI investigation of the effects of sequential instructional sets on cortical activity in a task‐switching paradigm , 2004, Human brain mapping.

[41]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[42]  S. Yantis,et al.  Selective visual attention and perceptual coherence , 2006, Trends in Cognitive Sciences.

[43]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.