Touch, sound and vision in human superior temporal sulcus

Human superior temporal sulcus (STS) is thought to be a key brain area for multisensory integration. Many neuroimaging studies have reported integration of auditory and visual information in STS but less is known about the role of STS in integrating other sensory modalities. In macaque STS, the superior temporal polysensory area (STP) responds to somatosensory, auditory and visual stimulation. To determine if human STS contains a similar area, we measured brain responses to somatosensory, auditory and visual stimuli using blood-oxygen level dependent functional magnetic resonance imaging (BOLD fMRI). An area in human posterior STS, STSms (multisensory), responded to stimulation in all three modalities. STSms responded during both active and passive presentation of unisensory somatosensory stimuli and showed larger responses for more intense vs. less intense tactile stimuli, hand vs. foot, and contralateral vs. ipsilateral tactile stimulation. STSms showed responses of similar magnitude for unisensory tactile and auditory stimulation, with an enhanced response to simultaneous auditory-tactile stimulation. We conclude that STSms is important for integrating information from the somatosensory as well as the auditory and visual modalities, and could be the human homolog of macaque STP.

[1]  T. Hackett,et al.  Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys , 2007, The Journal of comparative neurology.

[2]  W. K. Simmons,et al.  Measuring selectivity in fMRI data , 2007, Nature Neuroscience.

[3]  H Burton,et al.  Reading embossed capital letters: An fMRI study in blind and sighted individuals , 2006, Human brain mapping.

[4]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[5]  Jeffery A. Jones,et al.  Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information , 2004, Journal of Cognitive Neuroscience.

[6]  A. Puce,et al.  Neuronal oscillations and visual amplification of speech , 2008, Trends in Cognitive Sciences.

[7]  J. Kaiser,et al.  Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas , 2007, The Journal of Neuroscience.

[8]  B. Seltzer,et al.  Architectonics and cortical connections of the upper bank of the superior temporal sulcus in the rhesus monkey: An analysis in the tangential plane , 2003, The Journal of comparative neurology.

[9]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[10]  B. Argall,et al.  Simplified intersubject averaging on the cortical surface using SUMA , 2006, Human brain mapping.

[11]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[12]  John J. Foxe,et al.  Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. , 2002, Journal of neurophysiology.

[13]  D. V. van Essen,et al.  Mapping of architectonic subdivisions in the macaque monkey, with emphasis on parieto‐occipital cortex , 2000, The Journal of comparative neurology.

[14]  L. Krubitzer,et al.  Evidence for interhemispheric processing of inputs from the hands in human S2 and PV. , 2001, Journal of neurophysiology.

[15]  Lee M. Miller,et al.  Behavioral/systems/cognitive Perceptual Fusion and Stimulus Coincidence in the Cross- Modal Integration of Speech , 2022 .

[16]  Jack L. Lancaster,et al.  Clustered pixels analysis for functional MRI activation studies of the human brain , 1995 .

[17]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[18]  R. Adolphs.  Cognitive neuroscience: Cognitive neuroscience of human social behaviour , 2003, Nature Reviews Neuroscience.

[19]  Tony Ro,et al.  Human MST But Not MT Responds to Tactile Stimulation , 2007, The Journal of Neuroscience.

[20]  D. Heeger,et al.  Retinotopy and Functional Subdivision of Human Areas MT and MST , 2002, The Journal of Neuroscience.

[21]  S S Hsiao,et al.  Detection of vibration transmitted through an object grasped in the hand. , 1999, Journal of neurophysiology.

[22]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[23]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[24]  Paul J. Laurienti,et al.  On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies , 2005, Experimental Brain Research.

[25]  M. Wallace,et al.  Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. , 2005, Journal of neurophysiology.

[26]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[27]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[28]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[29]  R. Blake,et al.  Brain Areas Active during Visual Perception of Biological Motion , 2002, Neuron.

[30]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[31]  H. R. Clemo,et al.  Sensory and multisensory representations within the cat rostral suprasylvian cortex , 2007, The Journal of comparative neurology.

[32]  Gregory McCarthy,et al.  Polysensory interactions along lateral temporal regions evoked by audiovisual speech. , 2003, Cerebral cortex.

[33]  Stefan Golaszewski,et al.  Functional Magnetic Resonance Imaging of the Human Sensorimotor Cortex Using a Novel Vibrotactile Stimulator , 2002, NeuroImage.

[34]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[35]  Rainer Goebel,et al.  The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. , 2006, Cerebral cortex.

[36]  E Macaluso,et al.  Spatial and temporal factors during processing of audiovisual speech: a PET study , 2004, NeuroImage.

[37]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[38]  Riitta Hari,et al.  Audiovisual Integration of Letters in the Human Brain , 2000, Neuron.

[39]  E. Rolls,et al.  Functional subdivisions of the temporal lobe neocortex , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  Y. Sugita,et al.  Auditory-visual speech perception examined by fMRI and PET , 2003, Neuroscience Research.

[41]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[42]  Brian L Allman,et al.  Multisensory processing in "unimodal" neurons: cross-modal subthreshold auditory effects in cat extrastriate visual cortex. , 2007, Journal of neurophysiology.

[43]  C. Price.  The anatomy of language: contributions from functional neuroimaging , 2000, Journal of anatomy.

[44]  B. Stein,et al.  The Merging of the Senses , 1993 .

[45]  M. Hauser,et al.  The design of animal communication , 1999 .

[46]  T. Allison,et al.  Social perception from visual cues: role of the STS region , 2000, Trends in Cognitive Sciences.

[47]  G. Orban,et al.  Charting the Lower Superior Temporal Region, a New Motion-Sensitive Region in Monkey Superior Temporal Sulcus , 2006, The Journal of Neuroscience.

[48]  Istvan Ulbert,et al.  Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane , 2007, The Journal of comparative neurology.

[49]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[50]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[51]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.

[52]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[53]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[54]  Michael S. Beauchamp,et al.  Statistical criteria in fMRI studies of multisensory integration , 2005, Neuroinformatics.

[55]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[56]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[57]  Christoph Kayser,et al.  Functional organization of responses in the polysensory temporal region (STP) in the Macaque monkey , 2007 .

[58]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[59]  D C Van Essen,et al.  'The surface management system' (SuMS) database: a surface-based database to aid cortical surface reconstruction, visualization and analysis. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[60]  Nikos Makris,et al.  Automatically parcellating the human cerebral cortex. , 2004, Cerebral cortex.

[61]  I. Johnsrude,et al.  The problem of functional localization in the human brain , 2002, Nature Reviews Neuroscience.

[62]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[63]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[64]  Manabu Honda,et al.  Cross-modal binding and activated attentional networks during audio-visual speech integration: a functional MRI study. , 2005, Cerebral cortex.

[65]  Keiji Tanaka,et al.  Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. , 1988, Journal of neurophysiology.

[66]  Nancy Kanwisher,et al.  Divide and conquer: A defense of functional localizers , 2006, NeuroImage.