The Anterior N1 Component as an Index of Modality Shifting

Processing of a given target is facilitated when it is defined within the same (e.g., visual–visual), compared to a different (e.g., tactile–visual), perceptual modality as on the previous trial [Spence, C., Nicholls, M., & Driver, J. The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63, 330–336, 2001]. The present study was designed to identify electrocortical (EEG) correlates underlying this “modality shift effect.” Participants had to discriminate (via foot pedal responses) the modality of the target stimulus, visual versus tactile (Experiment 1), or respond based on the target-defining features (Experiment 2). Thus, modality changes were associated with response changes in Experiment 1, but dissociated in Experiment 2. Both experiments confirmed previous behavioral findings with slower discrimination times for modality change, relative to repetition, trials. Independently of the target-defining modality, spatial stimulus characteristics, and the motor response, this effect was mirrored by enhanced amplitudes of the anterior N1 component. These findings are explained in terms of a generalized “modality-weighting” account, which extends the “dimension-weighting” account proposed by Found and Müller [Searching for unknown feature targets on more than one dimension: Investigating a “dimension-weighting” account. Perception & Psychophysics, 58, 88–101, 1996] for the visual modality. On this account, the anterior N1 enhancement is assumed to reflect the detection of a modality change and initiation of the readjustment of attentional weight-setting from the old to the new target-defining modality in order to optimize target detection.

[1]  Martin Eimer,et al.  Crossmodal links in spatial attention are mediated by supramodal control processes: evidence from event-related potentials. , 2002, Psychophysiology.

[2]  E. Schröger,et al.  Measuring duration mismatch negativity , 2003, Clinical Neurophysiology.

[3]  Brigitte Röder,et al.  The redundant target effect is affected by modality switch costs , 2004, Psychonomic bulletin & review.

[4]  S Sutton,et al.  Evoked potential correlates of information processing in psychiatric patients , 1973, Psychological Medicine.

[5]  D. Siddle,et al.  Orienting and habituation : perspectives in human research , 1983 .

[6]  H J Müller,et al.  Visual search for singleton feature targets within and across feature dimensions , 1995, Perception & psychophysics.

[7]  Philip T Quinlan,et al.  Within- and between-dimensional processing in the auditory modality. , 2002, Journal of experimental psychology. Human perception and performance.

[8]  I. Winkler,et al.  Memory-based or afferent processes in mismatch negativity (MMN): a review of the evidence. , 2005, Psychophysiology.

[9]  T Kleinsorge,et al.  Response repetition benefits and costs. , 1999, Acta psychologica.

[10]  Effects of modality shift and motor response shift on simple reaction time in schizophrenia patients. , 1996, Journal of abnormal psychology.

[11]  H J Müller,et al.  Top-down controlled visual dimension weighting: an event-related fMRI study. , 2002, Cerebral cortex.

[12]  Erich Schröger,et al.  Response repetition vs. response change modulates behavioral and electrophysiological effects of distraction. , 2005, Brain research. Cognitive brain research.

[13]  Richard J. Brown,et al.  Investigating the nature and time-course of the modality shift effect between vision and touch , 2011, Quarterly journal of experimental psychology.

[14]  Joseph Krummenacher,et al.  Dimension‐specific intertrial facilitation in visual search for pop‐out targets: Evidence for a top‐down modulable visual short‐term memory effect , 2004 .

[15]  Stefan Pollmann,et al.  Anterior prefrontal cortex contributions to attention control. , 2004, Experimental psychology.

[16]  F. Rist,et al.  Effects of modality shift on event-related potentials and reaction time of chronic schizophrenics. , 1987, Electroencephalography and clinical neurophysiology. Supplement.

[17]  Neural temporal dynamics of contingency judgement , 2011, Quarterly journal of experimental psychology.

[18]  The Modality Shift Effect. Further Explorations at the Crossroads , 1992, Annals of the New York Academy of Sciences.

[19]  C. Rennie,et al.  Decrement of the N1 auditory event-related potential with stimulus repetition: habituation vs. refractoriness. , 1998, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[20]  Stefan Pollmann,et al.  Neural correlates of visual dimension weighting , 2006 .

[21]  H. Müller,et al.  Searching for unknown feature targets on more than one dimension: Investigating a “dimension-weighting” account , 1996, Perception & psychophysics.

[22]  S. Hochstein,et al.  The reverse hierarchy theory of visual perceptual learning , 2004, Trends in Cognitive Sciences.

[23]  C. Spence,et al.  The cost of expecting events in the wrong sensory modality , 2001, Perception & psychophysics.

[24]  Hermann J. Müller,et al.  Perceptual Basis of Redundancy Gains in Visual Pop-out Search , 2011, Journal of Cognitive Neuroscience.

[25]  Joseph Krummenacher,et al.  Brain electrical correlates of dimensional weighting: an ERP study. , 2007, Psychophysiology.

[26]  R. Verleger,et al.  Effects of certainty, modality shift and guess outcome on evoked potentials and reaction times in chronic schizophrenics , 1978, Psychological Medicine.

[27]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[28]  P. Rodway The modality shift effect and the effectiveness of warning signals in different modalities. , 2005, Acta psychologica.

[29]  E. Vogel,et al.  The visual N1 component as an index of a discrimination process. , 2000, Psychophysiology.

[30]  D. A. Nelson,et al.  Effects of intersignal interval on the human auditory evoked response. , 1968, The Journal of the Acoustical Society of America.

[31]  M. Eimer,et al.  Electrophysiological markers of visual dimension changes and response changes. , 2008, Journal of experimental psychology. Human perception and performance.

[32]  Yuping Wang,et al.  The sequential processing of visual feature conjunction mismatches in the human brain. , 2004, Psychophysiology.

[33]  J. Theeuwes,et al.  Response selection modulates visual search within and across dimensions. , 2005, Journal of experimental psychology. Human perception and performance.

[34]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[35]  Hermann J. Müller,et al.  Selective and interactive neural correlates of visual dimension changes and response changes , 2006, NeuroImage.

[36]  H. Müller,et al.  Top-down weighting of visual dimensions: Behavioral and electrophysiological evidence , 2010, Vision Research.

[37]  H. Müller,et al.  Dimension-based visual attention modulates dual-judgment accuracy in Duncan's (1984) one- versus two-object report paradigm. , 2000, Journal of experimental psychology. Human perception and performance.

[38]  M. Eimer,et al.  Modulations of early somatosensory ERP components by transient and sustained spatial attention , 2003, Experimental Brain Research.

[39]  K. Reinikainen,et al.  Attentive novelty detection in humans is governed by pre-attentive sensory memory , 1994, Nature.

[40]  Rolf Ulrich,et al.  The effect of a cross-trial shift of auditory warning signals on the sequential foreperiod effect. , 2010, Acta psychologica.

[41]  J. Driver,et al.  Crossmodal links in endogenous and exogenous spatial attention: evidence from event-related brain potential studies , 2001, Neuroscience & Biobehavioral Reviews.

[42]  H. J. Muller,et al.  Visual search for singleton feature targets across dimensions: Stimulus- and expectancy-driven effects in dimensional weighting. , 2003, Journal of experimental psychology. Human perception and performance.

[43]  S. Pollmann,et al.  A Fronto-Posterior Network Involved in Visual Dimension Changes , 2000, Journal of Cognitive Neuroscience.

[44]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[45]  S. Mannuzza Cross-modal reaction time and schizophrenic attentional deficit: a critical review. , 1980, Schizophrenia bulletin.

[46]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[47]  M. Farah,et al.  Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal? , 1989, Neuropsychologia.

[48]  H. Müller,et al.  Locus of dimension weighting: Preattentive or postselective? , 2006 .

[49]  Hermann J. Müller,et al.  Selective Visual Dimension Weighting Deficit after Left Lateral Frontopolar Lesions , 2007, Journal of Cognitive Neuroscience.

[50]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.