Auditory Motion Does Not Modulate Spiking Activity in Visual Motion Processing Areas MT and MST

The integration of multiple sensory modalities is one of the key aspects of brain function, allowing animals to take advantage of concurrent sources of information to make more accurate perceptual judgments. For many years, it was thought that multisensory integration in the cerebral cortex only occurs in high-level “polysensory” association areas, but recent studies have demonstrated cross-modal influences in regions that were traditionally designated as unimodal. In particular, several human neuroimaging studies have reported that extrastriate areas involved in visual motion perception are also activated by auditory motion, and may integrate audio-visual motion cues. However, the exact nature and extent of the effects of auditory motion on the visual cortex have not been studied at the single neuron level. We recorded the spiking activity of neurons in the middle temporal (MT) and medial superior temporal (MST) areas of anesthetized marmoset monkeys upon presentation of unimodal stimuli (moving auditory or visual patterns), as well as bimodal stimuli (concurrent audio-visual motion). Despite robust, direction selective responses to visual motion, none of the sampled neurons responded to auditory motion stimuli. Moreover, concurrent moving auditory stimuli had no significant effect on the ability of single MT and MST neurons, or populations of simultaneously recorded neurons, to discriminate the direction of motion of visual stimuli (moving random dot patterns with varying levels of motion noise). Our findings do not support the hypothesis that direct interactions between MT, MST and areas low in the hierarchy of auditory areas underlie audiovisual motion integration. Significance Statement Many studies have demonstrated that brain regions originally thought to be unisensory may play a role in multisensory processing. For example, some neuroimaging studies have found activity in regions involved in the processing of visual motion can be modified by auditory motion. We tested whether the spiking activity of neurons in two visual motion processing areas of the primate brain, areas MT and MST, can be modulated by moving auditory stimuli. Our results revealed that neurons in these areas neither respond to auditory motion, nor change their responses to visual motion according to auditory motion along the frontoparallel plane. These findings call into question the idea that audio-visual integration occurs at early stages of processing in the extrastriate cortex.

[1]  G. Essick,et al.  Tactile motion activates the human middle temporal/V5 (MT/V5) complex , 2002, The European journal of neuroscience.

[2]  Christoph Kayser,et al.  Sounds facilitate visual motion discrimination via the enhancement of late occipital visual representations , 2017, NeuroImage.

[3]  Brian L Allman,et al.  Do cross-modal projections always result in multisensory integration? , 2008, Cerebral cortex.

[4]  Brigitte Röder,et al.  Attending to visual or auditory motion affects perception within and across modalities: an event‐related potential study , 2005, The European journal of neuroscience.

[5]  Karen R Dobkins,et al.  The face inversion effect in infants is driven by high, and not low, spatial frequencies. , 2014, Journal of vision.

[6]  Giuliano Iurilli,et al.  Cellular and Synaptic Architecture of Multisensory Integration in the Mouse Neocortex , 2013, Neuron.

[7]  Mark W Greenlee,et al.  Neural correlates of coherent audiovisual motion perception. , 2007, Cerebral cortex.

[8]  Jack W. Tsao,et al.  Observed brain dynamics, P.P. Mitra, H. Bokil. Oxford University Press (2008), ISBN-13: 978-0-19-517808-1, 381 pages, $65.00 , 2009 .

[9]  H. R. Clemo,et al.  Laminar and connectional organization of a multisensory cortex , 2013, The Journal of comparative neurology.

[10]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[11]  Leo L. Lui,et al.  Responses of neurons in the marmoset primary auditory cortex to interaural level differences: comparison of pure tones and vocalizations , 2015, Front. Neurosci..

[12]  J. Rauschecker,et al.  Perception of Sound-Source Motion by the Human Brain , 2002, Neuron.

[13]  Chris Tailby,et al.  Visual motion integration by neurons in the middle temporal area of a New World monkey, the marmoset , 2011, The Journal of physiology.

[14]  S. Bensmaia,et al.  The neural basis of tactile motion perception. , 2014, Journal of neurophysiology.

[15]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[16]  S. Celebrini,et al.  Visuo-auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence , 2008, BMC Neuroscience.

[17]  Dora E Angelaki,et al.  Visual and Nonvisual Contributions to Three-Dimensional Heading Selectivity in the Medial Superior Temporal Area , 2006, The Journal of Neuroscience.

[18]  W. Newsome,et al.  Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[20]  Chia-Jung Chang,et al.  A quantitative acoustic analysis of the vocal repertoire of the common marmoset (Callithrix jacchus). , 2015, The Journal of the Acoustical Society of America.

[21]  Sean J. Slee,et al.  Sound localization cues in the marmoset monkey , 2010, Hearing Research.

[22]  I. Nelken,et al.  Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex , 2006, Cerebral cortex.

[23]  M A Meredith,et al.  Engagement of visual fixation suppresses sensory responsiveness and multisensory integration in the primate superior colliculus , 2003, The European journal of neuroscience.

[24]  Gene R. Stoner,et al.  Static sound timing alters sensitivity to low-level visual motion. , 2012, Journal of vision.

[25]  Jon Driver,et al.  Direction of Visual Apparent Motion Driven Solely by Timing of a Static Sound , 2008, Current Biology.

[26]  Frédéric Berthommier,et al.  Binding and unbinding the auditory and visual streams in the McGurk effect. , 2012, The Journal of the Acoustical Society of America.

[27]  A. Woods,et al.  Context Modulates the Contribution of Time and Space in Causal Inference , 2012, Front. Psychology.

[28]  C. Schroeder,et al.  How Local Is the Local Field Potential? , 2011, Neuron.

[29]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[30]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[31]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[32]  Michael S. Beauchamp,et al.  Re-examining overlap between tactile and visual motion responses within hMT+ and STS , 2015, NeuroImage.

[33]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[34]  Kaustubh Supekar,et al.  Distinct Global Brain Dynamics and Spatiotemporal Organization of the Salience Network , 2016, PLoS biology.

[35]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[36]  John H. R. Maunsell,et al.  Attentional modulation of visual motion processing in cortical areas MT and MST , 1996, Nature.

[37]  Megan A. K. Peters,et al.  0 + 1 > 1 , 2012, Psychological science.

[38]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[39]  J. Duhamel,et al.  Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey , 2007, The Journal of Neuroscience.

[40]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[41]  Christopher R Fetsch,et al.  Neural correlates of reliability-based cue weighting during multisensory integration , 2011, Nature Neuroscience.

[42]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[43]  A. King,et al.  Visual–auditory spatial processing in auditory cortical neurons , 2008, Brain Research.

[44]  W. Singer,et al.  Auditory motion direction encoding in auditory cortex and high‐level visual cortex , 2012, Human brain mapping.

[45]  C. Duffy MST neurons respond to optic flow and translational movement. , 1998, Journal of neurophysiology.

[46]  Brigitte Röder,et al.  Unimodal and crossmodal effects of endogenous attention to visual and auditory motion , 2004, Cognitive, affective & behavioral neuroscience.

[47]  Gregory C. DeAngelis,et al.  Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons , 2013, Nature Reviews Neuroscience.

[48]  Olivier White,et al.  The brain adjusts grip forces differently according to gravity and inertia: a parabolic flight experiment , 2015, Front. Integr. Neurosci..

[49]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[50]  Michael S. Osmanski,et al.  Measurement of absolute auditory thresholds in the common marmoset (Callithrix jacchus) , 2011, Hearing Research.

[51]  Philipp Berens,et al.  CircStat: AMATLABToolbox for Circular Statistics , 2009, Journal of Statistical Software.

[52]  M. Ernst,et al.  When Correlation Implies Causation in Multisensory Integration , 2012, Current Biology.

[53]  A. Caramazza,et al.  Multivoxel Pattern Analysis Reveals Auditory Motion Information in MT+ of Both Congenitally Blind and Sighted Individuals , 2013, PloS one.

[54]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[55]  G. Karmos,et al.  Entrainment of Neuronal Oscillations as a Mechanism of Attentional Selection , 2008, Science.

[56]  Ramesh Rajan,et al.  Auditory cortex of the marmoset monkey – complex responses to tones and vocalizations under opiate anaesthesia in core and belt areas , 2013, The European journal of neuroscience.

[57]  T. Stanford,et al.  Development of multisensory integration from the perspective of the individual neuron , 2014, Nature Reviews Neuroscience.

[58]  Marcello G P Rosa,et al.  Quantitative analysis of the corticocortical projections to the middle temporal area in the marmoset monkey: evolutionary and functional implications. , 2006, Cerebral cortex.

[59]  M. Wallace,et al.  Integration of multiple sensory modalities in cat cortex , 2004, Experimental Brain Research.

[60]  W. Singer,et al.  Capture of Auditory Motion by Vision Is Represented by an Activation Shift from Auditory to Visual Motion Cortex , 2008, The Journal of Neuroscience.

[61]  R. Rajan Centrifugal Pathways Protect Hearing Sensitivity at the Cochlea in Noisy Environments That Exacerbate the Damage Induced by Loud Sound , 2000, The Journal of Neuroscience.

[62]  A. Pouget,et al.  Efficient computation and cue integration with noisy population codes , 2001, Nature Neuroscience.

[63]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[64]  Leslie G. Ungerleider,et al.  Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque , 1990, The Journal of comparative neurology.

[65]  S. Lomber,et al.  Species-dependent role of crossmodal connectivity among the primary sensory cortices , 2017, Hearing Research.

[66]  G. Silberberg,et al.  Multisensory Integration in the Mouse Striatum , 2014, Neuron.

[67]  Maureen A. Hagan,et al.  Sensitivity of neurons in the middle temporal area of marmoset monkeys to random dot motion. , 2017, Journal of neurophysiology.

[68]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[69]  Leo L. Lui,et al.  Spatial and temporal frequency selectivity of neurons in the middle temporal visual area of new world monkeys (Callithrix jacchus) , 2007, The European journal of neuroscience.

[70]  G. Tononi,et al.  Consciousness and Anesthesia , 2008, Science.

[71]  Hans-Jochen Heinze,et al.  A movement-sensitive area in auditory cortex , 1999, Nature.

[72]  Giuliano Iurilli,et al.  Sound-Driven Synaptic Inhibition in Primary Visual Cortex , 2012, Neuron.

[73]  Rebecca Saxe,et al.  Sensitive Period for a Multimodal Response in Human Visual Motion Area MT/MST , 2010, Current Biology.

[74]  Leo L. Lui,et al.  Breaking camouflage: responses of neurons in the middle temporal area to stimuli defined by coherent motion , 2012, The European journal of neuroscience.

[75]  Ione Fine,et al.  Auditory motion processing after early blindness. , 2014, Journal of vision.

[76]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[77]  F. Gallyas Silver staining of myelin by means of physical development. , 1979, Neurological research.

[78]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[79]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[80]  Douglas A Ruff,et al.  Stimulus Dependence of Correlated Variability across Cortical Areas , 2016, The Journal of Neuroscience.

[81]  Brian Zingg,et al.  Cross-Modality Sharpening of Visual Cortical Processing through Layer-1-Mediated Inhibition and Disinhibition , 2016, Neuron.

[82]  Jennifer K Bizley,et al.  Where are multisensory signals combined for perceptual decision-making? , 2016, Current Opinion in Neurobiology.

[83]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[84]  Jennifer K. Bizley,et al.  Visual influences on ferret auditory cortex , 2009, Hearing Research.

[85]  Dora E Angelaki,et al.  Does the Middle Temporal Area Carry Vestibular Signals Related to Self-Motion? , 2009, The Journal of Neuroscience.

[86]  Leo L. Lui,et al.  Structure and function of the middle temporal visual area (MT) in the marmoset: Comparisons with the macaque monkey , 2015, Neuroscience Research.

[87]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[88]  C. Koch,et al.  The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes , 2012, Nature Reviews Neuroscience.

[89]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[90]  B. Stein,et al.  Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[91]  D. Heeger,et al.  Retinotopy and Functional Subdivision of Human Areas MT and MST , 2002, The Journal of Neuroscience.

[92]  W. Bair,et al.  Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior , 2001, The Journal of Neuroscience.

[93]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[94]  O. Andreassen,et al.  Mice Deficient in Cellular Glutathione Peroxidase Show Increased Vulnerability to Malonate, 3-Nitropropionic Acid, and 1-Methyl-4-Phenyl-1,2,5,6-Tetrahydropyridine , 2000, The Journal of Neuroscience.

[95]  Cyriel M A Pennartz,et al.  Audiovisual Modulation in Mouse Primary Visual Cortex Depends on Cross-Modal Stimulus Configuration and Congruency , 2017, The Journal of Neuroscience.

[96]  Hulusi Kafaligonul,et al.  Audiovisual associations alter the perception of low-level visual motion , 2015, Front. Integr. Neurosci..

[97]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[98]  Christoph M. Michel,et al.  Cortical Motion Deafness , 2004, Neuron.

[99]  Uta Noppeney,et al.  Sensory and Striatal Areas Integrate Auditory and Visual Signals into Behavioral Benefits during Motion Discrimination , 2013, The Journal of Neuroscience.

[100]  Bruce G Cumming,et al.  Feedforward and Feedback Sources of Choice Probability in Neural Population Responses This Review Comes from a Themed Issue on Neurobiology of Cognitive Behavior Evidence for Feed-forward Models and Optimal Linear Readout? , 2022 .

[101]  G. Elston,et al.  Visuotopic organisation and neuronal response selectivity for direction of motion in visual areas of the caudal temporal lobe of the marmoset monkey (Callithrix jacchus): Middle temporal area, middle temporal crescent, and surrounding cortex , 1998, The Journal of comparative neurology.

[102]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[103]  M. Rosa,et al.  A distinct anatomical network of cortical areas for analysis of motion in far peripheral vision , 2006, The European journal of neuroscience.

[104]  S. Zeki,et al.  Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. , 1971, Brain research.

[105]  Emiliano Ricciardi,et al.  Touching Motion: rTMS on the Human Middle Temporal Complex Interferes with Tactile Speed Perception , 2012, Brain Topography.

[106]  Alan Kingstone,et al.  Assessing automaticity in the audiovisual integration of motion. , 2005, Acta psychologica.

[107]  W. Singer,et al.  Auditory Motion Capturing Ambiguous Visual Motion , 2012, Front. Psychology.

[108]  Y. Cohen,et al.  The what, where and how of auditory-object perception , 2013, Nature Reviews Neuroscience.

[109]  Julie M. Harris,et al.  Optimal integration of shading and binocular disparity for depth perception. , 2012, Journal of vision.

[110]  P. Barone,et al.  Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey , 2005, The European journal of neuroscience.

[111]  P. Latham,et al.  Cracking the Neural Code for Sensory Perception by Combining Statistics, Intervention, and Behavior , 2017, Neuron.

[112]  G F Meyer,et al.  The integration of auditory and visual motion signals at threshold , 2003, Perception & psychophysics.

[113]  Benjamin A. Rowland,et al.  Multisensory Integration in the Superior Colliculus Requires Synergy among Corticocollicular Inputs , 2009, The Journal of Neuroscience.

[114]  John W Morley,et al.  Local and Global Correlations between Neurons in the Middle Temporal Area of Primate Visual Cortex. , 2015, Cerebral cortex.

[115]  S. Zeki Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey , 1974, The Journal of physiology.

[116]  Brian L Allman,et al.  Single‐unit analysis of somatosensory processing in the core auditory cortex of hearing ferrets , 2015, The European journal of neuroscience.

[117]  T. Albright Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[118]  W. Newsome,et al.  Estimates of the Contribution of Single Neurons to Perception Depend on Timescale and Noise Correlation , 2009, The Journal of Neuroscience.

[119]  A. Vanlierde,et al.  Specific activation of the V5 brain area by auditory motion processing: an fMRI study. , 2005, Brain research. Cognitive brain research.

[120]  E. Macaluso,et al.  A Common Cortical Substrate Activated by Horizontal and Vertical Sound Movement in the Human Brain , 2002, Current Biology.

[121]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[122]  M. Wallace,et al.  A revised view of sensory cortical parcellation , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[123]  Elise G. Rowe,et al.  Rapid Adaptation Induces Persistent Biases in Population Codes for Visual Motion , 2016, The Journal of Neuroscience.

[124]  D. J. Felleman,et al.  Receptive-field properties of neurons in middle temporal visual area (MT) of owl monkeys. , 1984, Journal of neurophysiology.

[125]  Marcia Grabowecky,et al.  Peripheral sounds rapidly activate visual cortex: evidence from electrocorticography. , 2015, Journal of neurophysiology.

[126]  U. Noppeney,et al.  Audiovisual Synchrony Improves Motion Discrimination via Enhanced Connectivity between Early Visual and Auditory Areas , 2010, The Journal of Neuroscience.

[127]  M A Meredith,et al.  The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. , 2001, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[128]  Keiji Tanaka,et al.  Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. , 1988, Journal of neurophysiology.

[129]  S. Wuerger,et al.  Cross-modal integration of auditory and visual motion signals , 2001, Neuroreport.

[130]  David Alais,et al.  No direction-specific bimodal facilitation for audiovisual motion detection. , 2004, Brain research. Cognitive brain research.

[131]  Tony Ro,et al.  Human MST But Not MT Responds to Tactile Stimulation , 2007, The Journal of Neuroscience.

[132]  P. Goldman-Rakic,et al.  Preface: Cerebral Cortex Has Come of Age , 1991 .

[133]  Frédéric Berthommier,et al.  Audio-visual speech scene analysis: characterization of the dynamics of unbinding and rebinding the McGurk effect. , 2015, The Journal of the Acoustical Society of America.

[134]  E. Rolls,et al.  Functional subdivisions of the temporal lobe neocortex , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[135]  Aaron R. Seitz,et al.  Sound Facilitates Visual Learning , 2006, Current Biology.