Similar categorical representation from sound and sight in the occipito-temporal cortex of sighted and blind

Is vision necessary for the development of the categorical organization of the Ventral Occipito-Temporal Cortex (VOTC)? We used fMRI to characterize VOTC responses to eight categories presented acoustically in sighted and early blind individuals, and visually in a separate sighted group. We observed that VOTC reliably encodes sound categories in sighted and blind people using a representational structure and connectivity partially similar to the one found in vision. Sound categories were, however, more reliably encoded in the blind than the sighted group, using a representational format closer to the one found in vision. Crucially, VOTC in blind represents the categorical membership of sounds rather than their acoustic features. Our results suggest that sounds trigger categorical responses in the VOTC of congenitally blind and sighted people that partially match the topography and functional profile of the visual response, despite qualitative nuances in the categorical organization of VOTC between modalities and groups.

[1]  Hans Op de Beeck,et al.  The (dis)similarities between neural networks based upon functional connectivity, representational similarity, and univariate analyses , 2018, bioRxiv.

[2]  K. Nakayama,et al.  RESPONSE PROPERTIES OF THE HUMAN FUSIFORM FACE AREA , 2000, Cognitive neuropsychology.

[3]  Nancy Kanwisher,et al.  Connectivity precedes function in the development of the visual word form area , 2016, Nature Neuroscience.

[4]  Radoslaw Martin Cichy,et al.  Probing principles of large‐scale object representation: Category preference and location encoding , 2013, Human brain mapping.

[5]  R. Malach,et al.  Cortical activity during tactile exploration of objects in blind and sighted humans. , 2010, Restorative neurology and neuroscience.

[6]  Jong Doo Lee,et al.  Morphological alterations in the congenital blind based on the analysis of cortical thickness and surface area , 2009, NeuroImage.

[7]  N. Kanwisher Functional specificity in the human brain: A window into the functional architecture of the mind , 2010, Proceedings of the National Academy of Sciences.

[8]  Talma Hendler,et al.  Center–periphery organization of human object areas , 2001, Nature Neuroscience.

[9]  Alexis Amadon,et al.  Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding , 2016, NeuroImage.

[10]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[11]  Miguel Marain-Padilla The Human Brain , 2011 .

[12]  O. Collignon,et al.  Neuronal populations in the occipital cortex of the blind synchronize to the temporal dynamics of speech , 2017, bioRxiv.

[13]  Tom Hartley,et al.  Low-Level Image Properties of Visual Objects Predict Patterns of Neural Response across Category-Selective Regions of the Ventral Visual Pathway , 2014, The Journal of Neuroscience.

[14]  Christian Büchel,et al.  Cortical hierarchy turned on its head , 2003, Nature Neuroscience.

[15]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[16]  Andreas Kleinschmidt,et al.  Interaction of Face and Voice Areas during Speaker Recognition , 2005, Journal of Cognitive Neuroscience.

[17]  A. Caramazza,et al.  Domain Selectivity in the Parahippocampal Gyrus Is Predicted by the Same Structural Connectivity Patterns in Blind and Sighted Individuals , 2017, The Journal of Neuroscience.

[18]  J. Rauschecker,et al.  Cortical Representation of Natural Complex Sounds: Effects of Acoustic Features and Auditory Object Category , 2010, The Journal of Neuroscience.

[19]  A. Caramazza,et al.  Neural representation of visual concepts in people born blind , 2018, Nature Communications.

[20]  Sheng He,et al.  Similarity representation of pattern-information fMRI , 2013 .

[21]  A. Snyder,et al.  Diffusion tensor imaging reveals white matter reorganization in early blind humans. , 2006, Cerebral cortex.

[22]  Bruce D. McCandliss,et al.  The visual word form area: expertise for reading in the fusiform gyrus , 2003, Trends in Cognitive Sciences.

[23]  Silvia Bernardini,et al.  The WaCky wide web: a collection of very large linguistically processed web-crawled corpora , 2009, Lang. Resour. Evaluation.

[24]  M. Hallett,et al.  Neural networks for Braille reading by the blind. , 1998 .

[25]  Bradford Z. Mahon,et al.  What drives the organization of object knowledge in the brain? , 2011, Trends in Cognitive Sciences.

[26]  S. Kosslyn,et al.  Topographical representations of mental images in primary visual cortex , 1995, Nature.

[27]  Emiliano Ricciardi,et al.  Beyond sensory images: Object-based representation in the human ventral pathway. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[28]  J. Rauschecker,et al.  A Positron Emission Tomographic Study of Auditory Localization in the Congenitally Blind , 2000, The Journal of Neuroscience.

[29]  R. Malach,et al.  Early ‘visual’ cortex activation correlates with superior verbal memory performance in the blind , 2003, Nature Neuroscience.

[30]  N. Kanwisher,et al.  The Human Body , 2001 .

[31]  N. Kanwisher,et al.  How face perception unfolds over time , 2018, Nature Communications.

[32]  Marius V Peelen,et al.  Shape-independent object category responses revealed by MEG and fMRI decoding. , 2016, Journal of neurophysiology.

[33]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[34]  Alfred Anwander,et al.  Direct Structural Connections between Voice- and Face-Recognition Areas , 2011, The Journal of Neuroscience.

[35]  Zeynep M. Saygin,et al.  Anatomical connectivity patterns predict face-selectivity in the fusiform gyrus , 2011, Nature Neuroscience.

[36]  J. Haxby,et al.  The effect of visual experience on the development of functional architecture in hMT+. , 2007, Cerebral cortex.

[37]  Emiliano Ricciardi,et al.  How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge , 2016, NeuroImage.

[38]  P. Downing,et al.  Category selectivity in human visual cortex: Beyond visual object recognition , 2017, Neuropsychologia.

[39]  O. Collignon,et al.  Functional selectivity in sensory-deprived cortices. , 2011, Journal of neurophysiology.

[40]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[41]  R Todd Constable,et al.  Image distortion correction in EPI: Comparison of field mapping with point spread function mapping , 2002, Magnetic resonance in medicine.

[42]  Daphne Bavelier,et al.  Human brain plasticity: evidence from sensory deprivation and altered language experience. , 2002, Progress in brain research.

[43]  Cortical Activity , 2020, Encyclopedia of Behavioral Medicine.

[44]  Miguel Marín-Padilla,et al.  The Human Brain , 2010 .

[45]  K M O'Craven,et al.  Structural and functional brain asymmetries in human situs inversus totalis , 1999, Neurology.

[46]  S. Kosslyn,et al.  Visual mental imagery induces retinotopically organized activation of early visual areas. , 2005, Cerebral cortex.

[47]  Nancy Kanwisher,et al.  An algorithmic method for functionally defining regions of interest in the ventral visual pathway , 2012, NeuroImage.

[48]  D. Bavelier,et al.  Cross-modal plasticity: where and how? , 2002, Nature Reviews Neuroscience.

[49]  Viviana Betti,et al.  Cortical cores in network dynamics , 2018, NeuroImage.

[50]  Alfonso Caramazza,et al.  Tool Selectivity in Left Occipitotemporal Cortex Develops without Vision , 2013, Journal of Cognitive Neuroscience.

[51]  Alfonso Caramazza,et al.  Plasticity based on compensatory effector use in the association but not primary sensorimotor cortex of people born without hands , 2018, Proceedings of the National Academy of Sciences.

[52]  Daria Proklova,et al.  MEG sensor patterns reflect perceptual but not categorical similarity of animate and inanimate objects , 2018, NeuroImage.

[53]  Jorge Jovicich,et al.  White matter connectivity between occipital and temporal regions involved in face and voice processing in hearing and early deaf individuals , 2018, NeuroImage.

[54]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[55]  Alfonso Caramazza,et al.  Selectivity for large nonmanipulable objects in scene-selective visual cortex does not require visual experience , 2013, NeuroImage.

[56]  H Burton,et al.  Reading embossed capital letters: An fMRI study in blind and sighted individuals , 2006, Human brain mapping.

[57]  Franco Lepore,et al.  Auditory motion in the sighted and blind: Early visual deprivation triggers a large-scale imbalance between auditory and “visual” brain regions , 2016, NeuroImage.

[58]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[59]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[60]  Colline Poirier,et al.  Auditory motion processing in early blind subjects , 2004, Cognitive Processing.

[61]  H. O. D. Beeck,et al.  Development of visual category selectivity in ventral visual cortex does not require visual experience , 2017 .

[62]  M. Livingstone,et al.  A hierarchical, retinotopic proto-organization of the primate visual system at birth , 2017, eLife.

[63]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[64]  A. Caramazza,et al.  Object Domain and Modality in the Ventral Visual Pathway , 2016, Trends in Cognitive Sciences.

[65]  Hans Op de Beeck,et al.  The ventral visual pathway represents animal appearance rather than animacy, unlike human behavior and deep neural networks , 2018 .

[66]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[67]  R. Saxe,et al.  Language processing in the occipital cortex of congenitally blind adults , 2011, Proceedings of the National Academy of Sciences.

[68]  Li Su,et al.  A Toolbox for Representational Similarity Analysis , 2014, PLoS Comput. Biol..

[69]  Robert H. Logie,et al.  Characteristics of visual short-term memory , 1989 .

[70]  Amir Amedi,et al.  Origins of the specialization for letters and numbers in ventral occipitotemporal cortex , 2015, Trends in Cognitive Sciences.

[71]  Chunshui Yu,et al.  Thick Visual Cortex in the Early Blind , 2009, The Journal of Neuroscience.

[72]  Alex Clarke,et al.  Learning Warps Object Representations in the Ventral Temporal Cortex , 2016, Journal of Cognitive Neuroscience.

[73]  Lucie Hertz-Pannier,et al.  MRI and M/EEG studies of the White Matter Development in Human Fetuses and Infants: Review and Opinion , 2016, Brain plasticity.

[74]  Tom Hartley,et al.  Selectivity for low-level features of objects in the human ventral stream , 2010, NeuroImage.

[75]  L. Abbott,et al.  Effects of early postnatal ethanol intubation on GABAergic synaptic proteins. , 2002, Brain research. Developmental brain research.

[76]  David A. Tovar,et al.  Representational dynamics of object vision: the first 1000 ms. , 2013, Journal of vision.

[77]  H. P. Op de Beeck,et al.  Dissociations and Associations between Shape and Category Representations in the Two Visual Pathways , 2015, The Journal of Neuroscience.

[78]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Magdalena G. Wutte,et al.  Modality-Independent Coding of Spatial Layout in the Human Brain , 2011, Current Biology.

[80]  Morgan D Barense,et al.  Integrative and distinctive coding of visual and conceptual object features in the ventral visual stream , 2018, eLife.

[81]  I. Fine,et al.  Responses in area hMT+ reflect tuning for both auditory frequency and motion after blindness early in life , 2019, Proceedings of the National Academy of Sciences.

[82]  Timothy E. J. Behrens,et al.  Human connectomics , 2012, Current Opinion in Neurobiology.

[83]  Justin L. Vincent,et al.  Novel domain formation reveals proto-architecture in inferotemporal cortex , 2014, Nature Neuroscience.

[84]  R. Tootell,et al.  Thinking Outside the Box: Rectilinear Shapes Selectively Activate Scene-Selective Cortex , 2014, The Journal of Neuroscience.

[85]  Talma Hendler,et al.  Eccentricity Bias as an Organizing Principle for Human High-Order Object Areas , 2002, Neuron.

[86]  N. Sadato,et al.  The Brain Network Underlying the Recognition of Hand Gestures in the Blind: The Supramodal Role of the Extrastriate Body Area , 2014, The Journal of Neuroscience.

[87]  Fuchun Lin,et al.  Progressive atrophy in the optic pathway and visual cortex of early blind Chinese adults: A voxel-based morphometry magnetic resonance imaging study , 2007, NeuroImage.

[88]  Franco Lepore,et al.  Plasticity in Sensory Systems: Building the Brain in the Dark: Functional and Specific Crossmodal Reorganization in the Occipital Cortex of Blind Individuals , 2012 .

[89]  Bruno L. Giordano,et al.  Abstract encoding of auditory objects in cortical activity patterns. , 2013, Cerebral cortex.

[90]  G. Dehaene-Lambertz,et al.  The early development of brain white matter: A review of imaging studies in fetuses, newborns and infants , 2014, Neuroscience.

[91]  J. S. Guntupalli,et al.  Decoding neural representational spaces using multivariate pattern analysis. , 2014, Annual review of neuroscience.

[92]  Chris I. Baker,et al.  Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images , 2019, NeuroImage.

[93]  G. Vandewalle,et al.  Functional specialization for auditory–spatial processing in the occipital cortex of congenitally blind humans , 2011, Proceedings of the National Academy of Sciences.

[94]  Alice J. O'Toole,et al.  Partially Distributed Representations of Objects and Faces in Ventral Temporal Cortex , 2005, Journal of Cognitive Neuroscience.

[95]  Amir Amedi,et al.  Reading with Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in the Blind , 2012, Neuron.

[96]  A. Cowey,et al.  Early Auditory Processing in Area V5/MT+ of the Congenitally Blind Brain , 2013, The Journal of Neuroscience.

[97]  J. L. de la Pompa,et al.  A novel source of arterial valve cells linked to bicuspid aortic valve without raphe in mice , 2018, eLife.

[98]  K. Davis,et al.  Linking oligodendrocyte and myelin dysfunction to neurocircuitry abnormalities in schizophrenia , 2011, Progress in Neurobiology.

[99]  F. Lepore,et al.  Functional preference for object sounds but not for voices in the occipito-temporal cortex of early blind individuals , 2017, bioRxiv.

[100]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[101]  Dimitrios Pantazis,et al.  Multivariate pattern analysis of MEG and EEG: A comparison of representational structure in time and space , 2016, NeuroImage.

[102]  Hans P. Op de Beeck,et al.  Development of visual category selectivity in ventral visual cortex does not require visual experience , 2017, Proceedings of the National Academy of Sciences.

[103]  Klaas E. Stephan,et al.  The anatomical basis of functional localization in the cortex , 2002, Nature Reviews Neuroscience.

[104]  Tom M. Mitchell,et al.  Machine learning classifiers and fMRI: A tutorial overview , 2009, NeuroImage.

[105]  Elia Formisano,et al.  Processing of Natural Sounds in Human Auditory Cortex: Tonotopy, Spectral Tuning, and Relation to Voice Sensitivity , 2012, The Journal of Neuroscience.

[106]  S Lehéricy,et al.  The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients. , 2000, Brain : a journal of neurology.

[107]  L. Merabet,et al.  Development of the Visual Word Form Area Requires Visual Experience: Evidence from Blind Braille Readers , 2017, The Journal of Neuroscience.

[108]  Kalanit Grill-Spector,et al.  Extensive childhood experience with Pokémon suggests eccentricity drives organization of visual cortex , 2019, Nature Human Behaviour.

[109]  Franco Lepore,et al.  Recruitment of the occipital cortex by arithmetic processing follows computational bias in the congenitally blind , 2019, NeuroImage.

[110]  Radoslaw Martin Cichy,et al.  Imagery and perception share cortical representations of content and location. , 2012, Cerebral Cortex.

[111]  Hans P. Op de Beeck,et al.  Factors Determining Where Category-Selective Areas Emerge in Visual Cortex , 2019, Trends in Cognitive Sciences.

[112]  Jörn Diedrichsen,et al.  Reliability of dissimilarity measures for multi-voxel pattern analysis , 2016, NeuroImage.

[113]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[114]  Milos Judas,et al.  The development of the subplate and thalamocortical connections in the human foetal brain , 2010, Acta paediatrica.

[115]  S. Evans,et al.  Sign and Speech Share Partially Overlapping Conceptual Representations , 2019, Current Biology.

[116]  M. Bedny Evidence from Blindness for a Cognitively Pluripotent Cortex , 2017, Trends in Cognitive Sciences.

[117]  Bruno Rossion,et al.  Functional selectivity for face processing in the temporal voice area of early deaf individuals , 2017, Proceedings of the National Academy of Sciences.

[118]  Bryan R. Conroy,et al.  A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex , 2011, Neuron.

[119]  Carlo Baldassi,et al.  Shape Similarity, Better than Semantic Membership, Accounts for the Structure of Visual Object Representations in a Population of Monkey Inferotemporal Neurons , 2013, PLoS Comput. Biol..

[120]  M. Bedny,et al.  “Visual” Cortex of Congenitally Blind Adults Responds to Syntactic Movement , 2015, The Journal of Neuroscience.

[121]  Scott D. Slotnick,et al.  The Visual Word Form Area , 2013 .

[122]  Jeffrey M. Zacks,et al.  Searchlight analysis: Promise, pitfalls, and potential , 2013, NeuroImage.

[123]  L. Tyler,et al.  Representational Similarity Analysis Reveals Commonalities and Differences in the Semantic Processing of Words and Objects , 2013, The Journal of Neuroscience.

[124]  Haemy Lee Masson,et al.  Comparing the functional structure of neural networks from representational similarity analysis with those from functional connectivity and univariate analyses , 2018 .

[125]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[126]  J. Duncan,et al.  Top-Down Activation of Shape-Specific Population Codes in Visual Cortex during Mental Imagery , 2009, The Journal of Neuroscience.

[127]  S. Kosslyn,et al.  The role of area 17 in visual imagery: convergent evidence from PET and rTMS. , 1999, Science.

[128]  Daria Proklova,et al.  Disentangling Representations of Object Shape and Object Category in Human Visual Cortex: The Animate–Inanimate Distinction , 2016, Journal of Cognitive Neuroscience.

[129]  Amir Amedi,et al.  Visual Cortex Extrastriate Body-Selective Area Activation in Congenitally Blind People “Seeing” by Using Sounds , 2014, Current Biology.

[130]  R. Malach,et al.  The topography of high-order human object areas , 2002, Trends in Cognitive Sciences.

[131]  Natalia Y. Bilenko,et al.  The “Parahippocampal Place Area” Responds Preferentially to High Spatial Frequencies in Humans and Monkeys , 2011, PLoS biology.

[132]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[133]  A. Caramazza,et al.  How Visual Is the Visual Cortex? Comparing Connectional and Functional Fingerprints between Congenitally Blind and Sighted Individuals , 2015, The Journal of Neuroscience.

[134]  A. Caramazza,et al.  Nonvisual and Visual Object Shape Representations in Occipitotemporal Cortex: Evidence from Congenitally Blind and Sighted Adults , 2014, The Journal of Neuroscience.

[135]  F. Rösler,et al.  Speech processing activates visual cortex in congenitally blind humans , 2002, The European journal of neuroscience.