Audiovisual semantic interference and attention: evidence from the attentional blink paradigm.

In the present study we investigate the role of attention in audiovisual semantic interference, by using an attentional blink paradigm. Participants were asked to make an unspeeded response to the identity of a visual target letter. This target letter was preceded at various SOAs by a synchronized audiovisual letter-pair, which was either congruent (e.g. hearing an "F" and viewing an "F") or incongruent (e.g. hearing an "F" and viewing a "Z"). In Experiment 1, participants were asked to match the members of the audiovisual letter-pair. In Experiment 2, participants were asked to ignore the synchronized audiovisual letter-pairs altogether and only report the visual target. In Experiment 3, participants were asked to identify only one of the audiovisual letters (identify the auditory letter, and ignore the synchronized visual letter, or vice versa). An attentional blink was found in all three experiments indicating that the audiovisual letter-pairs were processed. However, a congruency effect on subsequent target detection was observed in Experiments 1 and 3, but not in Experiment 2. The results indicate that attention to the semantic contents of at least one modality is necessary to establish audiovisual semantic interference.

[1]  S. Tremblay,et al.  Auditory attentional blink: masking the second target is necessary, delayed masking is sufficient. , 2005, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[2]  M C Potter,et al.  Two attentional deficits in serial target search: the visual attentional blink and an amodal task-switch deficit. , 1998, Journal of experimental psychology. Learning, memory, and cognition.

[3]  J. Theeuwes,et al.  Poke and pop: Tactile–visual synchrony increases visual saliency , 2009, Neuroscience Letters.

[4]  Bernhard Hommel,et al.  Short-term memory and the attentional blink: Capacity versus content , 2005, Memory & cognition.

[5]  R. Campbell,et al.  Audiovisual Integration of Speech Falters under High Attention Demands , 2005, Current Biology.

[6]  C. Olivers,et al.  Bleeping you out of the blink: Sound saves vision from oblivion , 2008, Brain Research.

[7]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[8]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[9]  John Duncan,et al.  Within-modality and cross-modality attentional blinks in a simple discrimination task , 2006, Perception & psychophysics.

[10]  C. Olivers,et al.  A boost and bounce theory of temporal attention. , 2008, Psychological review.

[11]  Kimron L. Shapiro,et al.  Does failure to mask T1 cause lag-1 sparing in the attentional blink? , 2008, Perception & psychophysics.

[12]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[13]  Charles Spence,et al.  Multisensory cues capture spatial attention regardless of perceptual load. , 2007, Journal of experimental psychology. Human perception and performance.

[14]  S. Tremblay,et al.  Modality-specific and amodal sources of interference in the attentional blink , 2008, Perception & psychophysics.

[15]  M. Potter,et al.  Temporal constraints on conscious vision: on the ubiquitous nature of the attentional blink. , 2009, Journal of vision.

[16]  Rainer Goebel,et al.  Top–down task effects overrule automatic multisensory responses to letter–sound pairs in auditory association cortex , 2006, NeuroImage.

[17]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[18]  M. Potter,et al.  Unmasking the attentional blink. , 2009, Journal of experimental psychology. Human perception and performance.

[19]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[20]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[21]  Nelson Cowan,et al.  The nature of cross-modal color-word interference effects , 1998, Perception & psychophysics.

[22]  B. Stein,et al.  Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis , 1996, Journal of Cognitive Neuroscience.

[23]  S. Tremblay,et al.  Delayed masking and the auditory attentional blink: A test for retrieval competition and bottleneck models. , 2006, Experimental psychology.

[24]  M. Masson,et al.  Using confidence intervals in within-subject designs , 1994, Psychonomic bulletin & review.

[25]  Andrew B. Leber,et al.  Made you blink! Contingent attentional capture produces a spatial blink , 2002, Perception & psychophysics.

[26]  Jan Theeuwes,et al.  The absence of an auditory-visual attentional blink is not due to echoic memory , 2007, Perception & psychophysics.

[27]  Karen M. Arnell,et al.  Cross-modality attentional blinks without preparatory task-set switching , 2002, Psychonomic bulletin & review.

[28]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[29]  John Duncan,et al.  Restricted attentional capacity within but not between sensory modalities , 1997, Nature.

[30]  Bernhard Hommel,et al.  Direct evidence for a role of working memory in the attentional blink , 2007, Memory & cognition.

[31]  K. Arnell,et al.  The attentional blink across stimulus modalities: Evidence for central processing limitations. , 1999 .

[32]  G. J. Thomas.  Experimental study of the influence of vision on sound localization. , 1941 .

[33]  William S Maki,et al.  Attentional capture triggers an attentional blink , 2006, Psychonomic bulletin & review.

[34]  E. Van der Burg,et al.  Audiovisual events capture attention: evidence from temporal order judgments. , 2008, Journal of vision.

[35]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[36]  Rainer Goebel,et al.  Visual influences on speech sound discrimination: A parametric fMRI study , 2006 .

[37]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[38]  Paul J. Laurienti,et al.  Semantic congruence is a critical factor in multisensory behavioral performance , 2004, Experimental Brain Research.

[39]  Dylan M. Jones,et al.  Attentional and perceptual sources of the auditory attentional blink , 2005, Perception & psychophysics.

[40]  J. Theeuwes,et al.  Priming T2 in a visual and auditory attentional blink task , 2006, Perception & psychophysics.

[41]  J. Navarra,et al.  Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task , 2004, Cognition.

[42]  Shin'ya Nishida,et al.  Visual search for a target changing in synchrony with an auditory signal , 2006, Proceedings of the Royal Society B: Biological Sciences.

[43]  John J. Foxe,et al.  Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. , 2004, Cerebral cortex.

[44]  J. Stroop.  Studies of interference in serial verbal reactions. , 1992 .

[45]  N Cowan,et al.  Cross-modal, auditory-visual Stroop interference and possible implications for speech memory , 1987, Perception & psychophysics.

[46]  Salvador Soto-Faraco,et al.  Attention to touch weakens audiovisual speech integration , 2007, Experimental Brain Research.

[47]  J. Hulleman,et al.  Spreading the sparing: against a limited-capacity account of the attentional blink , 2007, Psychological research.

[48]  Charles L. Folk,et al.  A Dissociation Between Attention and Selection , 2001, Psychological science.

[49]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[50]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[51]  L. Pessoa,et al.  Decoding near-threshold perception of fear from distributed single-trial brain activation. , 2006, Cerebral cortex.

[52]  Karen M. Arnell,et al.  Revisiting within-modality and cross-modality attentional blinks: Effects of target—distractor similarity , 2004, Perception & psychophysics.

[53]  M. Potter,et al.  Temporal Limits of Selection and Memory Encoding , 2006, Psychological science.

[54]  N. Kanwisher.  Repetition blindness: Type recognition without token individuation , 1987, Cognition.