Auditory and tactile frequency representations are co-embedded in modality-defined cortical sensory systems

Sensory information is represented and elaborated in hierarchical cortical systems that are thought to be dedicated to individual sensory modalities. This traditional view of sensory cortex organization has been challenged by recent evidence of multimodal responses in primary and association sensory areas. Although it is indisputable that sensory areas respond to multiple modalities, it remains unclear whether these multimodal responses reflect selective information processing for particular stimulus features. Here, we used fMRI adaptation to identify brain regions that are sensitive to the temporal frequency information contained in auditory, tactile, and audiotactile stimulus sequences. A number of brain regions distributed over the parietal and temporal lobes exhibited frequency-selective temporal response modulation for both auditory and tactile stimulus events, as indexed by repetition suppression effects. A smaller set of regions responded to crossmodal adaptation sequences in a frequency-dependent manner. Despite an extensive overlap of multimodal frequency-selective responses across the parietal and temporal lobes, representational similarity analysis revealed a cortical "regional landscape" that clearly reflected distinct somatosensory and auditory processing systems that converged on modality-invariant areas. These structured relationships between brain regions were also evident in spontaneous signal fluctuation patterns measured at rest. Our results reveal that multimodal processing in human cortex can be feature-specific and that multimodal frequency representations are embedded in the intrinsically hierarchical organization of cortical sensory systems.

[1]  Angela D Friederici,et al.  � Human Brain Mapping 24:11–20(2005) � Voice Perception: Sex, Pitch, and the Right Hemisphere , 2022 .

[2]  R. Veit,et al.  BOLD adaptation in vibrotactile stimulation: neuronal networks involved in frequency discrimination. , 2007, Journal of neurophysiology.

[3]  Ingo R. Titze,et al.  Principles of voice production , 1994 .

[4]  Daniel Bendor,et al.  Differential neural coding of acoustic flutter within primate auditory cortex , 2007, Nature Neuroscience.

[5]  J. F. Dammann,et al.  Temporal Frequency Channels Are Linked across Audition and Touch , 2009, Current Biology.

[6]  J. Yau,et al.  Reciprocal Interactions Between Audition and Touch in Flutter Frequency Perception. , 2019, Multisensory research.

[7]  Jarmo A. Hämäläinen,et al.  Repetition suppression comprises both attention-independent and attention-dependent processes , 2014, NeuroImage.

[8]  Amir Amedi,et al.  Origins of task-specific sensory-independent organization in the visual and auditory brain: neuroscience evidence, open questions and clinical implications , 2015, Current Opinion in Neurobiology.

[9]  D. Ferrington,et al.  Differential contributions to coding of cutaneous vibratory information by cortical somatosensory areas I and II. , 1980, Journal of neurophysiology.

[10]  P. Barone,et al.  Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey , 2005, The European journal of neuroscience.

[11]  J Mazziotta,et al.  A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[12]  Ranulfo Romo,et al.  A Neural Parametric Code for Storing Information of More than One Sensory Modality in Working Memory , 2016, Neuron.

[13]  Raymond J. Dolan,et al.  Selective Attention Modulates Neural Substrates of Repetition Priming and Implicit Visual Memory: Suppressions and Enhancements Revealed by fMRI , 2005, Journal of Cognitive Neuroscience.

[14]  S. Murray,et al.  Disrupted neural adaptation in autism spectrum disorder , 2018, bioRxiv.

[15]  Patrik Vuilleumier,et al.  Effects of attention and emotion on repetition priming and their modulation by cholinergic enhancement. , 2003, Journal of neurophysiology.

[16]  Riitta Hari,et al.  Touch activates human auditory cortex , 2006, NeuroImage.

[17]  M. Hollins,et al.  Vibrotactile adaptation enhances amplitude discrimination. , 1993, The Journal of the Acoustical Society of America.

[18]  M. Hollins,et al.  Vibrotactile adaptation enhances frequency discrimination. , 1994, The Journal of the Acoustical Society of America.

[19]  Hang Joon Jo,et al.  Mapping sources of correlation in resting state FMRI, with artifact detection and removal , 2010, NeuroImage.

[20]  Jeffrey M. Yau,et al.  Separate Mechanisms for Audio-Tactile Pitch and Loudness Interactions , 2010, Front. Psychology.

[21]  Jeffrey M Yau,et al.  Somatosensory interactions reveal feature-dependent computations. , 2019, Journal of neurophysiology.

[22]  Céline Cappe,et al.  The Thalamocortical Projection Systems in Primate: An Anatomical Support for Multisensory and Sensorimotor Interplay , 2009, Cerebral cortex.

[23]  Alex M. Kale,et al.  Reduced auditory cortical adaptation in autism spectrum disorder , 2018, eLife.

[24]  D. Huber,et al.  Feature-selective encoding of substrate vibrations in the forelimb somatosensory cortex , 2019, Nature.

[25]  Barak A. Pearlmutter,et al.  Illusory percepts from auditory adaptation. , 2007, The Journal of the Acoustical Society of America.

[26]  A. Mouraux,et al.  Primary sensory cortices contain distinguishable spatial patterns of activity for each sense , 2013, Nature Communications.

[27]  G. Rainer,et al.  Cognitive neuroscience: Neural mechanisms for detecting and remembering novel events , 2003, Nature Reviews Neuroscience.

[28]  David Alais,et al.  Auditory frequency perception adapts rapidly to the immediate past , 2015, Attention, perception & psychophysics.

[29]  Anders M. Dale,et al.  Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature , 2010, NeuroImage.

[30]  Felix Blankenburg,et al.  Tactile Motion and Pattern Processing Assessed with High-Field fMRI , 2011, PloS one.

[31]  Nikolaus Kriegeskorte,et al.  Visual representations are dominated by intrinsic fluctuations correlated between areas , 2015, NeuroImage.

[32]  D. Schacter,et al.  Cortical activity reductions during repetition priming can result from rapid response learning , 2004, Nature.

[33]  G. McCarthy,et al.  Perceiving patterns in random series: dynamic processing of sequence in prefrontal cortex , 2002, Nature Neuroscience.

[34]  Jonathan Winawer,et al.  Compressive Temporal Summation in Human Visual Cortex , 2017, The Journal of Neuroscience.

[35]  Jens Schwarzbach,et al.  The Contribution of Primary and Secondary Somatosensory Cortices to the Representation of Body Parts and Body Sides: An fMRI Adaptation Study , 2012, Journal of Cognitive Neuroscience.

[36]  G. Boynton,et al.  Adaptation: from single cells to BOLD signals , 2006, Trends in Neurosciences.

[37]  Rogelio Luna,et al.  Do Sensory Cortices Process More than One Sensory Modality during Perceptual Judgments? , 2010, Neuron.

[38]  Scott T. Grafton,et al.  Reductions in neural activity underlie behavioral components of repetition priming , 2005, Nature Neuroscience.

[39]  Christopher D. Chambers,et al.  Current perspectives and methods in studying neural mechanisms of multisensory interactions , 2012, Neuroscience & Biobehavioral Reviews.

[40]  Á. Pascual-Leone,et al.  The metamodal organization of the brain. , 2001, Progress in brain research.

[41]  Hannes P. Saal,et al.  Multiplexing Stimulus Information through Rate and Temporal Codes in Primate Somatosensory Cortex , 2013, PLoS biology.

[42]  Hannes P. Saal,et al.  Natural scenes in tactile texture. , 2014, Journal of neurophysiology.

[43]  S. Solomon,et al.  Moving Sensory Adaptation beyond Suppressive Effects in Single Neurons , 2014, Current Biology.

[44]  D. Bendor,et al.  The neuronal representation of pitch in primate auditory cortex , 2005, Nature.

[45]  Jochen Kaiser,et al.  Audiovisual Functional Magnetic Resonance Imaging Adaptation Reveals Multisensory Integration Effects in Object-Related Sensory Cortices , 2010, The Journal of Neuroscience.

[46]  S. Dehaene,et al.  The priming method: imaging unconscious repetition priming reveals an abstract representation of number in the parietal lobes. , 2001, Cerebral cortex.

[47]  B L Whitsel,et al.  Time course and action spectrum of vibrotactile adaptation. , 1990, Somatosensory & motor research.

[48]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[49]  J. Diedrichsen,et al.  Hand use predicts the structure of representations in sensorimotor cortex , 2015, Nature Neuroscience.

[50]  Jörn Diedrichsen,et al.  Pattern component modeling: A flexible approach for understanding the representational structure of brain activity patterns , 2017, NeuroImage.

[51]  S. Lederman Auditory Texture Perception , 1979, Perception.

[52]  J. Hyvärinen,et al.  Cortical neuronal mechanisms in flutter-vibration studied in unanesthetized monkeys. Neuronal periodicity and frequency discrimination. , 1969, Journal of neurophysiology.

[53]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[54]  Vincent Hayward,et al.  Sensing with tools extends somatosensory processing beyond the body , 2018, Nature.

[55]  Alex Martin,et al.  Individual differences in intrinsic brain connectivity predict decision strategy. , 2014, Journal of neurophysiology.

[56]  Guillaume A. Rousselet,et al.  Improving standards in brain-behavior correlation analyses , 2012, Front. Hum. Neurosci..

[57]  John C. Mazziotta,et al.  A Probabilistic Atlas and Reference System for the Human Brain , 2001 .

[58]  L. Merabet,et al.  Neural reorganization following sensory loss: the opportunity of change , 2010, Nature Reviews Neuroscience.

[59]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[60]  S. Rose Selective attention , 1992, Nature.

[61]  R. Romo,et al.  Periodicity and Firing Rate As Candidate Neural Codes for the Frequency of Vibrotactile Stimuli , 2000, The Journal of Neuroscience.

[62]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[63]  Kerry M. M. Walker,et al.  Neural Mechanisms for the Abstraction and Use of Pitch Information in Auditory Cortex , 2012, The Journal of Neuroscience.

[64]  M. Tommerdahl,et al.  Human vibrotactile frequency discriminative capacity after adaptation to 25 Hz or 200 Hz stimulation , 2005, Brain Research.

[65]  J. Andrew Pruszynski,et al.  BOLD Responses to Tactile Stimuli in Visual and Auditory Cortex Depend on the Frequency Content of Stimulation , 2012, Journal of Cognitive Neuroscience.

[66]  Tony Ro,et al.  A neural link between feeling and hearing. , 2013, Cerebral cortex.

[67]  John J. Foxe,et al.  Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. , 2002, Journal of neurophysiology.

[68]  E. M. Rouiller,et al.  Multisensory anatomical pathways , 2009, Hearing Research.

[69]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[70]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[71]  Nikolaus Kriegeskorte,et al.  Face-identity change activation outside the face system: "release from adaptation" may not always indicate neuronal selectivity. , 2010, Cerebral cortex.

[72]  Amir Amedi,et al.  Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach , 2009, Experimental Brain Research.

[73]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[74]  Helen C. Barron,et al.  Repetition suppression: a means to index neural representations using BOLD? , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[75]  Georgios A Keliris,et al.  Neurons in macaque area V4 acquire directional tuning after adaptation to motion stimuli , 2005, Nature Neuroscience.

[76]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[77]  M. Hollins,et al.  Vibrotactile intensity and frequency information in the Pacinian system: A psychophysical model , 2005, Perception & psychophysics.

[78]  Xiaoqin Wang,et al.  Importance of spike timing in touch: an analogy with hearing? , 2016, Current Opinion in Neurobiology.

[79]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[80]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[81]  Md. Shoaibur Rahman,et al.  Selective Attention Gates the Interactive Crossmodal Coupling between Perceptual Systems , 2017, Current Biology.

[82]  Ankoor S. Shah,et al.  Auditory Cortical Neurons Respond to Somatosensory Stimulation , 2003, The Journal of Neuroscience.

[83]  Istvan Ulbert,et al.  Sources of Somatosensory Input to the Caudal Belt Areas of Auditory Cortex , 2007, Perception.

[84]  Alexis Pérez-Bellido,et al.  Auditory Frequency Representations in Human Somatosensory Cortex , 2018, Cerebral cortex.

[85]  S. Shimojo,et al.  Rate perception adapts across the senses: evidence for a unified timing mechanism , 2015, Scientific Reports.

[86]  Tony Ro,et al.  Neural Substrates of Sound–Touch Synesthesia after a Thalamic Lesion , 2008, The Journal of Neuroscience.

[87]  Lexi E. Crommett,et al.  Auditory adaptation improves tactile frequency perception. , 2017, Journal of neurophysiology.

[88]  Vincent Hayward,et al.  Motion Aftereffects Transfer between Touch and Vision , 2009, Current Biology.

[89]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[90]  E. Zwicker “Negative Afterimage” in Hearing , 1964 .

[91]  Joel L. Voss,et al.  Establishing a relationship between activity reduction in human perirhinal cortex and priming , 2009, Hippocampus.