Audio‐visual multisensory training enhances visual processing of motion stimuli in healthy participants: an electrophysiological study

Evidence from electrophysiological and imaging studies suggests that audio‐visual (AV) stimuli presented in spatial coincidence enhance activity in the subcortical colliculo‐dorsal extrastriate pathway. To test whether repetitive AV stimulation might specifically activate this neural circuit underlying multisensory integrative processes, electroencephalographic data were recorded before and after 2 h of AV training, during the execution of two lateralized visual tasks: a motion discrimination task, relying on activity in the colliculo‐dorsal MT pathway, and an orientation discrimination task, relying on activity in the striate and early ventral extrastriate cortices. During training, participants were asked to detect and perform a saccade towards AV stimuli that were disproportionally allocated to one hemifield (the trained hemifield). Half of the participants underwent a training in which AV stimuli were presented in spatial coincidence, while the remaining half underwent a training in which AV stimuli were presented in spatial disparity (32°). Participants who received AV training with stimuli in spatial coincidence had a post‐training enhancement of the anterior N1 component in the motion discrimination task, but only in response to stimuli presented in the trained hemifield. However, no effect was found in the orientation discrimination task. In contrast, participants who received AV training with stimuli in spatial disparity showed no effects on either task. The observed N1 enhancement might reflect enhanced discrimination for motion stimuli, probably due to increased activity in the colliculo‐dorsal MT pathway induced by multisensory training.

[1]  Nestor Matthews,et al.  Task-specific perceptual learning on speed and direction discrimination , 2003, Vision Research.

[2]  S. Petersen,et al.  The pulvinar and visual salience , 1992, Trends in Neurosciences.

[3]  Frank Tong,et al.  Relationship between BOLD amplitude and pattern classification of orientation-selective activity in the human visual cortex , 2012, NeuroImage.

[4]  M T Wallace,et al.  Mechanisms of within- and cross-modality suppression in the superior colliculus. , 1997, Journal of neurophysiology.

[5]  B. Stein,et al.  Adult Plasticity in Multisensory Neurons: Short-Term Experience-Dependent Changes in the Superior Colliculus , 2009, The Journal of Neuroscience.

[6]  G. Boynton,et al.  Orientation-Specific Adaptation in Human Visual Cortex , 2003, The Journal of Neuroscience.

[7]  Terrence J. Sejnowski,et al.  Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis , 2007, NeuroImage.

[8]  Nadia Bolognini,et al.  Audiovisual Integration in Patients with Visual Deficit , 2005, Journal of Cognitive Neuroscience.

[9]  Antigona Martínez,et al.  Source analysis of event-related cortical activity during visuo-spatial attention. , 2003, Cerebral cortex.

[10]  Jiang Qiu,et al.  The time course of visual categorization: Electrophysiological evidence from ERP , 2006 .

[11]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[12]  Donatella Spinelli,et al.  Spatiotemporal brain mapping of spatial attention effects on pattern‐reversal ERPs , 2012, Human brain mapping.

[13]  R. Wurtz,et al.  Functional Identification of a Pulvinar Path from Superior Colliculus to Cortical Area MT , 2010, The Journal of Neuroscience.

[14]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[15]  V. Romei,et al.  Crossmodal enhancement of visual orientation discrimination by looming sounds requires functional activation of primary visual areas: A case study , 2013, Neuropsychologia.

[16]  Fabrizio Leo,et al.  Multisensory integration for orienting responses in humans requires the activation of the superior colliculus , 2008, Experimental Brain Research.

[17]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: anatomic location and visual response properties. , 1993, Journal of neurophysiology.

[18]  S. Hillyard,et al.  Cortical sources of the early components of the visual evoked potential , 2002, Human brain mapping.

[19]  Caterina Bertini,et al.  Audio-visual stimulation improves oculomotor patterns in patients with hemianopia , 2009, Neuropsychologia.

[20]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[21]  Béatrice de Gelder,et al.  Exploring the relation between mcgurk interference and ventriloquism , 1994, ICSLP.

[22]  E Donchin,et al.  A new method for off-line removal of ocular artifact. , 1983, Electroencephalography and clinical neurophysiology.

[23]  E. Macaluso,et al.  Spatial orienting in complex audiovisual environments , 2014, Human brain mapping.

[24]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[25]  E. Chudler,et al.  Somatosensory, multisensory, and task-related neurons in cortical area 7b (PF) of unanesthetized monkeys. , 1994, Journal of neurophysiology.

[26]  G. Orban,et al.  The Retinotopic Organization of the Human Middle Temporal Area MT/V5 and Its Cortical Neighbors , 2010, The Journal of Neuroscience.

[27]  G. Pourtois,et al.  Top-down effects on early visual processing in humans: A predictive coding framework , 2011, Neuroscience & Biobehavioral Reviews.

[28]  A. King,et al.  The superior colliculus , 2004, Current Biology.

[29]  Robert H. Wurtz,et al.  Signals Conveyed in the Pulvinar Pathway from Superior Colliculus to Cortical Area MT , 2011, The Journal of Neuroscience.

[30]  Ning Qian,et al.  Learning motion discrimination with suppressed MT , 2004, Vision Research.

[31]  Nadia Bolognini,et al.  Visual search improvement in hemianopic patients after audio-visual stimulation , 2005 .

[32]  Daniel Kersten,et al.  Spatially specific FMRI repetition effects in human visual cortex. , 2006, Journal of neurophysiology.

[33]  Essa Yacoub,et al.  High-field fMRI unveils orientation columns in humans , 2008, Proceedings of the National Academy of Sciences.

[34]  R. Andersen,et al.  Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[35]  Benjamin A Rowland,et al.  Initiating the Development of Multisensory Integration by Manipulating Sensory Experience , 2010, The Journal of Neuroscience.

[36]  Benjamin A. Rowland,et al.  Incorporating Cross-Modal Statistics in the Development and Maintenance of Multisensory Integration , 2012, The Journal of Neuroscience.

[37]  G. Orban,et al.  Activity of inferior temporal neurons during orientation discrimination with successively presented gratings. , 1994, Journal of neurophysiology.

[38]  S. Hillyard,et al.  Involvement of striate and extrastriate visual cortical areas in spatial attention , 1999, Nature Neuroscience.

[39]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[40]  E. Vogel,et al.  The visual N1 component as an index of a discrimination process. , 2000, Psychophysiology.

[41]  Steven A. Hillyard,et al.  Identification of the neural sources of the pattern-reversal VEP , 2005, NeuroImage.

[42]  D. Hubel,et al.  Orientation columns in macaque monkey visual cortex demonstrated by the 2-deoxyglucose autoradiographic technique , 1977, Nature.

[43]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[44]  Jinghong Xu,et al.  Multisensory plasticity in adulthood: cross-modal experience enhances neuronal excitability and exposes silent inputs. , 2013, Journal of neurophysiology.

[45]  M. Posner,et al.  Frontal and inferior temporal cortical activity in visual target detection: Evidence from high spatially sampled event-related potentials , 1996, Brain Topography.

[46]  G. Pourtois,et al.  Effects of perceptual learning on primary visual cortex activity in humans , 2008, Vision Research.

[47]  Jascha D. Swisher,et al.  Multiscale Pattern Analysis of Orientation-Selective Activity in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[48]  R. Krauzlis,et al.  Superior colliculus and visual spatial attention. , 2013, Annual review of neuroscience.

[49]  Aaron R. Seitz,et al.  Benefits of Stimulus Congruency for Multisensory Facilitation of Visual Learning , 2008, PloS one.

[50]  Martin E. Maier,et al.  Multisensory stimulation in hemianopic patients boosts orienting responses to the hemianopic field and reduces attentional resources to the intact field. , 2015, Restorative neurology and neuroscience.

[51]  D. Kersten,et al.  Orientation-tuned FMRI adaptation in human visual cortex. , 2005, Journal of neurophysiology.

[52]  John H. R. Maunsell,et al.  The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[53]  S. Zeki Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey , 1974, The Journal of physiology.

[54]  E. Làdavas,et al.  When apperceptive agnosia is explained by a deficit of primary visual processing , 2014, Cortex.

[55]  N. Bolognini,et al.  Is audiovisual integration subserved by the superior colliculus in humans? , 2008, Neuroreport.

[56]  E Macaluso,et al.  Spatial and temporal factors during processing of audiovisual speech: a PET study , 2004, NeuroImage.

[57]  Jinghong Xu,et al.  Multisensory Plasticity in Superior Colliculus Neurons is Mediated by Association Cortex. , 2016, Cerebral cortex.

[58]  Fabrizio Leo,et al.  Independent mechanisms for ventriloquism and multisensory integration as revealed by theta‐burst stimulation , 2010, The European journal of neuroscience.

[59]  C. Spence Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule , 2013, Annals of the New York Academy of Sciences.

[60]  Frank Bremmer,et al.  Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP) , 2002, The European journal of neuroscience.

[61]  M. Fahle Perceptual learning: specificity versus generalization , 2005, Current Opinion in Neurobiology.

[62]  D. Tucker,et al.  Frontal evaluation and posterior representation in target detection. , 2001, Brain research. Cognitive brain research.

[63]  R J Snowden,et al.  Identification of Visual Stimuli is Improved by Accompanying Auditory Stimuli: The Role of Eye Movements and Sound Location , 2001, Perception.

[64]  Fabrizio Leo,et al.  Temporo-nasal asymmetry in multisensory integration mediated by the Superior Colliculus , 2008, Brain Research.

[65]  M. McCourt,et al.  Hemifield asymmetry in the potency of exogenous auditory and visual cues , 2011, Vision Research.

[66]  T. Albright Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[67]  C. Escera,et al.  Effects of sound location on visual task performance and electrophysiological measures of distraction , 2008, Neuroreport.

[68]  Edward M. Callaway,et al.  A Disynaptic Relay from Superior Colliculus to Dorsal Stream Visual Cortex in Macaque Monkey , 2010, Neuron.

[69]  Diego Pinal,et al.  Effects of load and maintenance duration on the time course of information encoding and retrieval in working memory: from perceptual analysis to post-categorization processes , 2014, Front. Hum. Neurosci..

[70]  W. Singer,et al.  Retinotopic effects during spatial audio-visual integration , 2007, Neuropsychologia.

[71]  Richard S. J. Frackowiak,et al.  Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. , 1993, Cerebral cortex.

[72]  T. Stanford,et al.  Noise‐rearing disrupts the maturation of multisensory integration , 2014, The European journal of neuroscience.

[73]  Sabine Kastner,et al.  Visual responses of the human superior colliculus: a high-resolution functional magnetic resonance imaging study. , 2005, Journal of neurophysiology.

[74]  J. Ygge,et al.  Motion perception in children with foetal alcohol syndrome , 2006, Acta paediatrica.

[75]  Mark E. McCourt,et al.  Biases of spatial attention in vision and audition , 2010, Brain and Cognition.

[76]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[77]  B. Stein,et al.  The Merging of the Senses , 1993 .

[78]  J. K. Harting,et al.  Ascending pathways from the monkey superior colliculus: An autoradiographic analysis , 1980, The Journal of comparative neurology.

[79]  G. Potts An ERP index of task relevance evaluation of visual stimuli , 2004, Brain and Cognition.

[80]  Leslie G. Ungerleider,et al.  Microsaccadic eye movements and firing of single cells in the striate cortex of macaque monkeys , 2000, Nature Neuroscience.

[81]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[82]  P. Mamassian,et al.  Multisensory processing in review: from physiology to behaviour. , 2010, Seeing and perceiving.

[83]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[84]  Lin Yang,et al.  Perceptual Learning Increases the Strength of the Earliest Signals in Visual Cortex , 2010, The Journal of Neuroscience.

[85]  B. Stein,et al.  Multisensory training reverses midbrain lesion-induced changes and ameliorates haemianopia , 2015, Nature Communications.

[86]  S. Hillyard,et al.  Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. , 1991, Journal of experimental psychology. Human perception and performance.