Attentional Resource Allocation in Visuotactile Processing Depends on the Task, But Optimal Visuotactile Integration Does Not Depend on Attentional Resources

Humans constantly process and integrate sensory input from multiple sensory modalities. However, the amount of input that can be processed is constrained by limited attentional resources. A matter of ongoing debate is whether attentional resources are shared across sensory modalities, and whether multisensory integration is dependent on attentional resources. Previous research suggested that the distribution of attentional resources across sensory modalities depends on the the type of tasks. Here, we tested a novel task combination in a dual task paradigm: Participants performed a self-terminated visual search task and a localization task in either separate sensory modalities (i.e., haptics and vision) or both within the visual modality. Tasks considerably interfered. However, participants performed the visual search task faster when the localization task was performed in the tactile modality in comparison to performing both tasks within the visual modality. This finding indicates that tasks performed in separate sensory modalities rely in part on distinct attentional resources. Nevertheless, participants integrated visuotactile information optimally in the localization task even when attentional resources were diverted to the visual search task. Overall, our findings suggest that visual search and tactile localization partly rely on distinct attentional resources, and that optimal visuotactile integration is not dependent on attentional resources.

[1]  P. König,et al.  Beyond sensory substitution—learning the sixth sense , 2005, Journal of neural engineering.

[2]  Salvador Soto-Faraco,et al.  Top-down attention regulates the neural expression of audiovisual integration , 2015, NeuroImage.

[3]  Mahdi Nezamabadi,et al.  Color Appearance Models , 2014, J. Electronic Imaging.

[4]  Marty G. Woldorff,et al.  Selective Attention and Multisensory Integration: Multiple Phases of Effects on the Evoked Brain Activity , 2005, Journal of Cognitive Neuroscience.

[5]  M C Potter,et al.  Two attentional deficits in serial target search: the visual attentional blink and an amodal task-switch deficit. , 1998, Journal of experimental psychology. Learning, memory, and cognition.

[6]  M. Sams,et al.  Effect of attentional load on audiovisual speech perception: evidence from ERPs , 2014, Front. Psychol..

[7]  Andreas K. Engel,et al.  Oscillatory signatures of crossmodal congruence effects: An EEG investigation employing a visuotactile pattern matching paradigm , 2015, NeuroImage.

[8]  Thomas L. Thornton,et al.  Parallel and serial processes in visual search. , 2007, Psychological review.

[9]  Karen M. Arnell,et al.  Revisiting within-modality and cross-modality attentional blinks: Effects of target—distractor similarity , 2004, Perception & psychophysics.

[10]  P. König,et al.  Where's the action? The pragmatic turn in cognitive science , 2013, Trends in Cognitive Sciences.

[11]  E. Macaluso,et al.  Processing of multisensory spatial congruency can be dissociated from working memory and visuo‐spatial attention , 2007, The European journal of neuroscience.

[12]  Kerry Hourigan,et al.  Wake transition of a rolling sphere , 2011, J. Vis..

[13]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[14]  Heath A. Ruff,et al.  Utilty of a Tactile Display for Cueing Faults , 2002 .

[15]  H. Ehrsson,et al.  Disintegration of Multisensory Signals from the Real Hand Reduces Default Limb Self-Attribution: An fMRI Study , 2013, The Journal of Neuroscience.

[16]  Peter König,et al.  Sensory Augmentation for the Blind , 2012, Front. Hum. Neurosci..

[17]  Yuli D. Chashechkin,et al.  Transportation of a dye in upstream and downstream wakes of the cylinder in continuously stratified liquid , 2007, J. Vis..

[18]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[19]  D. Burr,et al.  Vision and Audition Do Not Share Attentional Resources in Sustained Tasks , 2011, Front. Psychology.

[20]  C. Rorden,et al.  Covert orienting of attention and overt eye movements activate identical brain regions , 2008, Brain Research.

[21]  Andreas K. Engel,et al.  Attention Modulates Visual-Tactile Interaction in Spatial Pattern Matching , 2014, PloS one.

[22]  Peter König,et al.  Audition and vision share spatial attentional resources, yet attentional load does not disrupt audiovisual integration , 2015, Front. Psychol..

[23]  N. Lavie Distracted and confused?: Selective attention under load , 2005, Trends in Cognitive Sciences.

[24]  Julie D. Golomb,et al.  A taxonomy of external and internal attention. , 2011, Annual review of psychology.

[25]  C. Spence Crossmodal spatial attention , 2010, Annals of the New York Academy of Sciences.

[26]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[27]  Salvador Soto-Faraco,et al.  Attention to touch weakens audiovisual speech integration , 2007, Experimental Brain Research.

[28]  David Alais,et al.  Separate attentional resources for vision and audition , 2006, Proceedings of the Royal Society B: Biological Sciences.

[29]  C. Spence,et al.  Modality-specific auditory and visual temporal processing deficits , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[30]  J. Enns,et al.  Spatial selection and target identification are separable processes in visual search. , 2010, Journal of vision.

[31]  David Goldblatt,et al.  Brain, Mind, and Behavior , 1985 .

[32]  D. Spalding The Principles of Psychology , 1873, Nature.

[33]  Mark D. Fairchild,et al.  Color Appearance Models: Fairchild/Color Appearance Models , 2013 .

[34]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[35]  S. Gepshtein,et al.  The combination of vision and touch depends on spatial proximity. , 2005, Journal of vision.

[36]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[37]  Sabine U. König,et al.  The experience of new sensorimotor contingencies by sensory augmentation , 2014, Consciousness and Cognition.

[38]  Marc O Ernst,et al.  Visual-haptic cue weighting is independent of modality-specific attention. , 2008, Journal of vision.

[39]  P. König,et al.  Multisensory teamwork: using a tactile or an auditory display to exchange gaze information improves performance in joint visual search , 2016, Ergonomics.

[40]  Emiliano Macaluso,et al.  Orienting of spatial attention and the interplay between the senses , 2010, Cortex.

[41]  W. Gorski,et al.  Visualization of cavities on ship screw propeller , 2002 .

[42]  A. Matzger Where's the Action? , 1971, California medicine.

[43]  George A Alvarez,et al.  How many objects can you track? Evidence for a resource-limited attentive tracking mechanism. , 2007, Journal of vision.

[44]  W Prinzmetal,et al.  Dividing attention between the color and the shape of objects , 1998, Perception & psychophysics.

[45]  M Corbetta,et al.  Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[46]  P. Bertelson,et al.  The ventriloquist effect does not depend on the direction of automatic visual attention , 2001, Perception & psychophysics.

[47]  M. Woldorff,et al.  Attentional capacity for processing concurrent stimuli is larger across sensory modalities than within a modality. , 2006, Psychophysiology.

[48]  Charles Spence,et al.  Cross-modal congruency and visual capture in a visual elevation-discrimination task , 2003, Experimental Brain Research.

[49]  P. König,et al.  Vision and Haptics Share Spatial Attentional Resources and Visuotactile Integration Is Not Affected by High Attentional Load. , 2015, Multisensory research.

[50]  Markus H. Sneve,et al.  Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. , 2014, Journal of vision.

[51]  M. Eimer The neural basis of attentional control in visual search , 2014, Trends in Cognitive Sciences.

[52]  Charles Spence,et al.  Multisensory cues capture spatial attention regardless of perceptual load. , 2007, Journal of experimental psychology. Human perception and performance.

[53]  Marc O. Ernst,et al.  A Bayesian view on multimodal cue integration , 2006 .

[54]  D. Talsma Predictive coding and multisensory integration: an attentional account of the multisensory mind , 2015, Front. Integr. Neurosci..

[55]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[56]  Jason S. Chan,et al.  Behavioral evidence for task-dependent “what” versus “where” processing within and across modalities , 2008, Perception & psychophysics.

[57]  R. Campbell,et al.  Audiovisual Integration of Speech Falters under High Attention Demands , 2005, Current Biology.

[58]  John Duncan,et al.  Within-modality and cross-modality attentional blinks in a simple discrimination task , 2006, Perception & psychophysics.

[59]  C. Spence,et al.  A crossmodal attentional blink between vision and touch , 2002, Psychonomic bulletin & review.

[60]  Nadine B. Sarter,et al.  Good Vibrations: Tactile Feedback in Support of Attention Allocation and Human-Automation Coordination in Event-Driven Domains , 1999, Hum. Factors.

[61]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[62]  P. Jolicoeur Restricted attentional capacity between sensory modalities , 1999, Psychonomic bulletin & review.

[63]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[64]  Paul J. Laurienti,et al.  Modality-specific selective attention attenuates multisensory integration , 2007, Experimental Brain Research.

[65]  C. Spence,et al.  Spatial constraints on visual-tactile cross-modal distractor congruency effects , 2004, Cognitive, affective & behavioral neuroscience.

[66]  Jan Theeuwes,et al.  The absence of an auditory-visual attentional blink is not due to echoic memory , 2007, Perception & psychophysics.

[67]  Karen M. Arnell,et al.  Cross-modality attentional blinks without preparatory task-set switching , 2002, Psychonomic bulletin & review.