Cross-modal decoupling in temporal attention between audition and touch

Temporal orienting leads to well-documented behavioural benefits for sensory events occurring at the anticipated moment. However, the consequences of temporal orienting in cross-modal contexts are still unclear. On the one hand, some studies using audio-tactile paradigms suggest that attentional orienting in time and modality are a closely coupled system, in which temporal orienting dominates modality orienting, similar to what happens in cross-modal spatial attention. On the other hand, recent findings using a visuo-tactile paradigm suggest that attentional orienting in time can unfold independently in each modality, leading to cross-modal decoupling. In the present study, we investigated if cross-modal decoupling in time can be extrapolated to audio-tactile contexts. If so, decoupling might represent a general property of cross-modal attention in time. To this end, we used a speeded discrimination task in which we manipulated the probability of target presentation in time and modality. In each trial, a manipulation of time-based expectancy was used to guide participants’ attention to task-relevant events, either tactile or auditory, at different points in time. In two experiments, we found that participants generally showed enhanced behavioural performance at the most likely onset time of each modality and no evidence for coupling. This pattern supports the hypothesis that cross-modal decoupling could be a general phenomenon in temporal orienting.

[1]  J. Driver,et al.  Attentional selection of location and modality in vision and touch modulates low-frequency activity in associated sensory cortices , 2012, Journal of neurophysiology.

[2]  Jinglong Wu,et al.  An fMRI Study of the Neural Systems Involved in Visually Cued Auditory Top-Down Spatial and Temporal Attention , 2012, PloS one.

[3]  O. Jensen,et al.  Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition , 2010, Front. Hum. Neurosci..

[4]  C. Spence,et al.  Attention and the crossmodal construction of space , 1998, Trends in Cognitive Sciences.

[5]  G. Dreisbach,et al.  The time-event correlation effect is due to temporal expectancy, not to partial transition costs. , 2015, Journal of experimental psychology. Human perception and performance.

[6]  Rolf Ulrich,et al.  Task predictability influences the variable foreperiod effect: evidence of task-specific temporal preparation , 2015, Psychological research.

[7]  Salvador Soto-Faraco,et al.  The COGs (context, object, and goals) in multisensory processing , 2016, Experimental Brain Research.

[8]  Hauke R. Heekeren,et al.  What Happens in Between? Human Oscillatory Brain Activity Related to Crossmodal Spatial Cueing , 2008, PloS one.

[9]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[10]  Lisa D. Sanders,et al.  Temporally selective attention modulates early perceptual processing: Event-related potential evidence , 2008, Perception & psychophysics.

[11]  K. Lange Brain correlates of early auditory processing are attenuated by expectations for time and pitch , 2009, Brain and Cognition.

[12]  John J. Foxe,et al.  Intersensory selective attention and temporal orienting operate in parallel and are instantiated in spatially distinct sensory and motor cortices , 2015, Human brain mapping.

[13]  Janneke F. M. Jehee,et al.  Attention Reverses the Effect of Prediction in Silencing Sensory Signals , 2011, Cerebral cortex.

[14]  Jinglong Wu,et al.  The interactions of multisensory integration with endogenous and exogenous attention , 2016, Neuroscience & Biobehavioral Reviews.

[15]  B. Milliken,et al.  Endogenous temporal orienting of attention in detection and discrimination tasks , 2004, Perception & psychophysics.

[16]  S. Kotz,et al.  Dissociation of formal and temporal predictability in early auditory evoked potentials , 2013, Neuropsychologia.

[17]  A. Nobre,et al.  Spatial and temporal acuity of visual perception can be enhanced selectively by attentional set , 2008, Experimental Brain Research.

[18]  S. Treue Visual attention: the where, what, how and why of saliency , 2003, Current Opinion in Neurobiology.

[19]  Martin Eimer,et al.  Can attention be directed to opposite locations in different modalities? An ERP study , 1999, Clinical Neurophysiology.

[20]  Salvador Soto-Faraco,et al.  Alpha Stimulation of the Human Parietal Cortex Attunes Tactile Perception to External Space , 2014, Current Biology.

[21]  Viola S. Störmer,et al.  Cross-modal cueing of attention alters appearance and early cortical processing of visual stimuli , 2009, Proceedings of the National Academy of Sciences.

[22]  G. Mangun,et al.  Top-down Modulation of Neural Activity in Anticipatory Visual Attention: Control Mechanisms Revealed by Simultaneous EEG-fMRI. , 2014, Cerebral cortex.

[23]  Gregor Volberg,et al.  Time-based expectations entail preparatory motor activity , 2017, Cortex.

[24]  Andrea Kiesel,et al.  Transfer of time-based task expectancy across different timing environments , 2018, Psychological research.

[25]  A. Nobre,et al.  Time for the Fourth Dimension in Attention , 2014 .

[26]  C Miniussi,et al.  Orienting attention in time. , 2001, Frontiers in bioscience : a journal and virtual library.

[27]  S. Soto-Faraco,et al.  Cross‐modal decoupling in temporal attention , 2014, The European journal of neuroscience.

[28]  Gesine Dreisbach,et al.  Time-based event expectations employ relative, not absolute, representations of time , 2015, Psychonomic bulletin & review.

[29]  J. Lupiáñez,et al.  Attentional preparation based on temporal expectancy modulates processing at the perceptual level , 2005, Psychonomic bulletin & review.

[30]  Brigitte Röder,et al.  Early processing stages are modulated when auditory stimuli are presented at an attended moment in time: an event-related potential study. , 2003, Psychophysiology.

[31]  A. Nobre,et al.  Temporal Expectation Improves the Quality of Sensory Information , 2012, The Journal of Neuroscience.

[32]  Rufin VanRullen,et al.  On the cyclic nature of perception in vision versus audition , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  Jon Driver,et al.  Cross-Modal Interactions between Audition, Touch, and Vision in Endogenous Spatial Attention: ERP Evidence on Preparatory States and Sensory Modulations , 2002, Journal of Cognitive Neuroscience.

[34]  J. Driver,et al.  Audiovisual links in endogenous covert spatial attention. , 1996, Journal of experimental psychology. Human perception and performance.

[35]  C. Spence,et al.  The cost of expecting events in the wrong sensory modality , 2001, Perception & psychophysics.

[36]  Michael Kubovy,et al.  Should We Resist the Seductiveness of the Space:Time::Vision:Audition Analogy? , 1988 .

[37]  Emiliano Macaluso,et al.  Orienting of spatial attention and the interplay between the senses , 2010, Cortex.

[38]  A. Nobre,et al.  Orienting attention in time. Modulation of brain potentials. , 1999, Brain : a journal of neurology.

[39]  C. Spence,et al.  Tactile-Visual Links in Exogenous Spatial Attention under Different Postures: Convergent Evidence from Psychophysics and ERPs , 2001, Journal of Cognitive Neuroscience.

[40]  Franck Vidal,et al.  Orienting Attention in Time Activates Left Intraparietal Sulcus for Both Perceptual and Motor Task Goals , 2011, Journal of Cognitive Neuroscience.

[41]  J. Hoffmann,et al.  The scope and precision of specific temporal expectancy: evidence from a variable foreperiod paradigm , 2011, Attention, perception & psychophysics.

[42]  Alan Kingstone,et al.  On audiovisual spatial synergy: The fragility of the phenomenon , 2005, Perception & psychophysics.

[43]  K. Lange,et al.  Effects of temporal trial-by-trial cuing on early and late stages of auditory processing: Evidence from event-related potentials , 2011, Attention, perception & psychophysics.

[44]  A. Nobre,et al.  Temporal Expectation Enhances Contrast Sensitivity by Phase Entrainment of Low-Frequency Oscillations in Visual Cortex , 2013, The Journal of Neuroscience.

[45]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[46]  E. Schröger,et al.  Early electrophysiological indicators for predictive processing in audition: a review. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[47]  Brigitte Röder,et al.  Attending points in time and space , 2006, Experimental Brain Research.

[48]  J Driver,et al.  An event-related brain potential study of cross-modal links in spatial attention between vision and touch. , 2000, Psychophysiology.

[49]  Ulrich Pomper,et al.  Distinct patterns of local oscillatory activity and functional connectivity underlie intersensory attention and temporal prediction , 2016, Cortex.

[50]  Natasha Merat,et al.  Crossmodal links between audition and touch in covert endogenous spatial attention , 2003, Perception & psychophysics.

[51]  C. Spence,et al.  On measuring selective attention to an expected sensory modality , 1997, Perception & psychophysics.

[52]  Felix Ball,et al.  The role of multisensory interplay in enabling temporal expectations , 2017, Cognition.

[53]  K. Lange The ups and downs of temporal orienting: a review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction , 2013, Front. Hum. Neurosci..

[54]  E. Macaluso,et al.  Multisensory spatial interactions: a window onto functional integration in the human brain , 2005, Trends in Neurosciences.

[55]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[56]  Brigitte Röder,et al.  Orienting Attention to Points in Time Improves Stimulus Processing Both within and across Modalities , 2006, Journal of Cognitive Neuroscience.

[57]  Bettina Forster,et al.  When far is near: ERP correlates of crossmodal spatial interactions between tactile and mirror-reflected visual stimuli , 2011, Neuroscience Letters.

[58]  Emiliano Macaluso,et al.  The costs of monitoring simultaneously two sensory modalities decrease when dividing attention in space , 2010, NeuroImage.

[59]  Charles Spence,et al.  Spatial synergies between auditory and visual attention. , 1994 .

[60]  C. Summerfield,et al.  Expectation (and attention) in visual cognition , 2009, Trends in Cognitive Sciences.

[61]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[62]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[63]  M. Johnson,et al.  Circulating microRNAs in Sera Correlate with Soluble Biomarkers of Immune Activation but Do Not Predict Mortality in ART Treated Individuals with HIV-1 Infection: A Case Control Study , 2015, PloS one.

[64]  C. Frith,et al.  Modulation of human visual cortex by crossmodal spatial attention. , 2000, Science.

[65]  H. Spitzer,et al.  Increased attention enhances both behavioral and neuronal performance. , 1988, Science.

[66]  Joachim Hoffmann,et al.  Temporal cueing of target-identity and target-location. , 2010, Experimental psychology.

[67]  A. Nobre,et al.  Where and When to Pay Attention: The Neural Systems for Directing Attention to Spatial Locations and to Time Intervals as Revealed by Both PET and fMRI , 1998, The Journal of Neuroscience.

[68]  Andrew R Mayer,et al.  An event‐related FMRI study of exogenous orienting across vision and audition , 2014, Human brain mapping.

[69]  C. Spence,et al.  Crossmodal links between vision and touch in covert endogenous spatial attention. , 2000, Journal of experimental psychology. Human perception and performance.

[70]  C. Spence,et al.  Cross-modal links in spatial attention. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[71]  D. Knol,et al.  The foreperiod effect revisited: conditioning as a basis for nonspecific preparation. , 2001, Acta psychologica.

[72]  Julian Keil,et al.  Temporal orienting precedes intersensory attention and has opposing effects on early evoked brain activity , 2017, NeuroImage.

[73]  R. Klein Attention and visual dominance: a chronometric analysis. , 1977, Journal of experimental psychology. Human perception and performance.

[74]  Kathrin Lange,et al.  The N1 effect of temporal attention is independent of sound location and intensity: implications for possible mechanisms of temporal attention. , 2012, Psychophysiology.

[75]  Jan-Mathijs Schoffelen,et al.  Temporal Expectation and Attention Jointly Modulate Auditory Oscillatory Activity in the Beta Band , 2015, PloS one.

[76]  E. Maris,et al.  Orienting Attention to an Upcoming Tactile Event Involves a Spatially and Temporally Specific Modulation of Sensorimotor Alpha- and Beta-Band Oscillations , 2011, The Journal of Neuroscience.

[77]  S. Hillyard,et al.  Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study. , 1999, Brain research. Cognitive brain research.