A new method for detecting interactions between the senses in event-related potentials

Event-related potentials (ERPs) can be used in multisensory research to determine the point in time when different senses start to interact, for example, the auditory and the visual system. For this purpose, the ERP to bimodal stimuli (AV) is often compared to the sum of the ERPs to auditory (A) and visual (V) stimuli: AV - (A + V). If the result is non-zero, this is interpreted as an indicator for multisensory interactions. Using this method, several studies have demonstrated auditory-visual interactions as early as 50 ms after stimulus onset. The subtraction requires that A, V, and AV do not contain common activity: This activity would be subtracted twice from one ERP and would, therefore, contaminate the result. In the present study, ERPs to unimodal, bimodal, and trimodal auditory, visual, and tactile stimuli (T) were recorded. We demonstrate that (T + TAV) - (TA + TV) is equivalent to AV - (A + V), but common activity is eliminated because two ERPs are subtracted from two others. With this new comparison technique, the first auditory-visual interaction starts around 80 ms after stimulus onset for the present experimental setting. It is possible to apply the new comparison method to other brain imaging techniques, as well, e.g. functional magnetic resonance imaging.

[1]  S. Sterbing-D’Angelo,et al.  Behavioral/systems/cognitive Multisensory Space Representations in the Macaque Ventral Intraparietal Area , 2022 .

[2]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[3]  D. Raab.  Statistical facilitation of simple reaction times. , 1962, Transactions of the New York Academy of Sciences.

[4]  Juliana Yordanova,et al.  Spatial coincidence modulates interaction between visual and somatosensory evoked potentials , 2002, Neuroreport.

[5]  J. Pernier,et al.  Early auditory-visual interactions in human cortex during nonredundant target identification. , 2002, Brain research. Cognitive brain research.

[6]  Brigitte Röder,et al.  Multisensory processing in the redundant-target effect: A behavioral and event-related potential study , 2005, Perception & psychophysics.

[7]  A. Diederich,et al.  Probability inequalities for testing separate activation models of divided attention , 1992, Perception & psychophysics.

[8]  E. Schröger,et al.  Speeded responses to audiovisual signal changes result from bimodal integration. , 1998, Psychophysiology.

[9]  中村 道彦,et al.  Contingent Negative Variation〔邦文〕 (脳脊髄誘発電位の基礎と臨床 ) , 1979 .

[10]  W. Ritter,et al.  The scalp topography of potentials associated with missing visual or auditory stimuli. , 1976, Electroencephalography and clinical neurophysiology.

[11]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[12]  W. Walter,et al.  Contingent Negative Variation : An Electric Sign of Sensori-Motor Association and Expectancy in the Human Brain , 1964, Nature.

[13]  Riitta Hari,et al.  Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction , 2003, NeuroImage.

[14]  Laura Busse,et al.  The ERP omitted stimulus response to “no-stim” events and its implications for fast-rate event-related fMRI designs , 2003, NeuroImage.

[15]  S A Hillyard,et al.  An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. , 2002, Brain research. Cognitive brain research.

[16]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[17]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[18]  M. Wallace,et al.  Sensory and Multisensory Responses in the Newborn Monkey Superior Colliculus , 2001, The Journal of Neuroscience.

[19]  M. Wallace,et al.  Integration of multiple sensory modalities in cat cortex , 2004, Experimental Brain Research.

[20]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[21]  S. Sternberg,et al.  Separate modifiability, mental modules, and the use of pure and composite measures to reveal them. , 2001, Acta psychologica.

[22]  D. Raab.  DIVISION OF PSYCHOLOGY: STATISTICAL FACILITATION OF SIMPLE REACTION TIMES* , 1962 .

[23]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[24]  D. Barth,et al.  The spatiotemporal organization of auditory, visual, and auditory-visual evoked potentials in rat cortex , 1995, Brain Research.