Modality-independent encoding of individual concepts in the left parietal cortex

ABSTRACT The organization of semantic information in the brain has been mainly explored through category‐based models, on the assumption that categories broadly reflect the organization of conceptual knowledge. However, the analysis of concepts as individual entities, rather than as items belonging to distinct superordinate categories, may represent a significant advancement in the comprehension of how conceptual knowledge is encoded in the human brain. Here, we studied the individual representation of thirty concrete nouns from six different categories, across different sensory modalities (i.e., auditory and visual) and groups (i.e., sighted and congenitally blind individuals) in a core hub of the semantic network, the left angular gyrus, and in its neighboring regions within the lateral parietal cortex. Four models based on either perceptual or semantic features at different levels of complexity (i.e., low‐ or high‐level) were used to predict fMRI brain activity using representational similarity encoding analysis. When controlling for the superordinate component, high‐level models based on semantic and shape information led to significant encoding accuracies in the intraparietal sulcus only. This region is involved in feature binding and combination of concepts across multiple sensory modalities, suggesting its role in high‐level representation of conceptual knowledge. Moreover, when the information regarding superordinate categories is retained, a large extent of parietal cortex is engaged. This result indicates the need to control for the coarse‐level categorial organization when performing studies on higher‐level processes related to the retrieval of semantic information. HighlightsWe studied the semantic representation of concrete nouns in the left lateral parietal cortex.Four models were used to predict fMRI responses to individual nouns from different categories.Posterior and middle IPS retained high‐level perceptual and semantic information.Information about superordinate categories yielded a different recruitment of parietal regions.

[1]  Hervé Abdi,et al.  How the Human Brain Represents Perceived Dangerousness or “Predacity” of Animals , 2016, The Journal of Neuroscience.

[2]  P. Pietrini,et al.  Mind the blind brain to understand the sighted one! Is there a supramodal cortical functional architecture? , 2014, Neuroscience & Biobehavioral Reviews.

[3]  J. S. Guntupalli,et al.  The Representation of Biological Classes in the Human Brain , 2012, The Journal of Neuroscience.

[4]  L. Barsalou On Staying Grounded and Avoiding Quixotic Dead Ends , 2016, Psychonomic Bulletin & Review.

[5]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[6]  Benjamin D. Zinszer,et al.  Representational similarity encoding for fMRI: Pattern-based synthesis to predict brain activity using stimulus-model-similarities , 2016, NeuroImage.

[7]  L. Barsalou What does semantic tiling of the cortex tell us about semantics? , 2017, Neuropsychologia.

[8]  Karl J. Friston,et al.  Effects of visual deprivation on the organization of the semantic system. , 2003, Brain : a journal of neurology.

[9]  Juha Silvanto,et al.  The Causal Role of the Occipital Face Area (OFA) and Lateral Occipital (LO) Cortex in Symmetry Perception , 2015, The Journal of Neuroscience.

[10]  J. Binder In defense of abstract conceptual representations , 2016, Psychonomic Bulletin & Review.

[11]  John A. Pyles,et al.  Comparing visual representations across human fMRI and computational vision. , 2013, Journal of vision.

[12]  Cathy J. Price,et al.  Functional Heterogeneity within the Default Network during Semantic Processing and Speech Production , 2012, Front. Psychology.

[13]  M. Jenkinson,et al.  Non-linear optimisation FMRIB Technial Report TR 07 JA 1 , 2007 .

[14]  David J. Freedman,et al.  Experience-dependent representation of visual categories in parietal cortex , 2006, Nature.

[15]  Paul Hoffman,et al.  The Semantic Network at Work and Rest: Differential Connectivity of Anterior Temporal Lobe Subregions , 2016, The Journal of Neuroscience.

[16]  A. Treisman,et al.  The Interaction of Spatial and Object Pathways: Evidence from Balint's Syndrome , 1997, Journal of Cognitive Neuroscience.

[17]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[18]  L. Robertson Binding, spatial attention and perceptual awareness , 2003, Nature Reviews Neuroscience.

[19]  Carlo Baldassi,et al.  Shape Similarity, Better than Semantic Membership, Accounts for the Structure of Visual Object Representations in a Population of Monkey Inferotemporal Neurons , 2013, PLoS Comput. Biol..

[20]  S. Kastner,et al.  Two hierarchically organized neural systems for object information in human visual cortex , 2008, Nature Neuroscience.

[21]  N. Kriegeskorte,et al.  Visual features as stepping stones toward semantics: Explaining object similarity in IT and perception with non-negative least squares , 2015, Neuropsychologia.

[22]  Guido Gainotti,et al.  The influence of anatomical locus of lesion and of gender-related familiarity factors in category-specific semantic disorders for animals, fruits and vegetables: A review of single-case studies , 2010, Cortex.

[23]  L. Barsalou,et al.  Perceptual simulation in conceptual combination: evidence from property generation. , 2009, Acta psychologica.

[24]  Nikolaus Kriegeskorte,et al.  The Emergence of Semantic Meaning in the Ventral Temporal Pathway , 2014, Journal of Cognitive Neuroscience.

[25]  H. Blum Biological shape and visual science (part I) , 1973 .

[26]  A. Caramazza,et al.  The role of vision in the neural representation of unique entities , 2016, Neuropsychologia.

[27]  A. Caramazza,et al.  Nonvisual and Visual Object Shape Representations in Occipitotemporal Cortex: Evidence from Congenitally Blind and Sighted Adults , 2014, The Journal of Neuroscience.

[28]  L. Tyler,et al.  Understanding What We See: How We Derive Meaning From Vision , 2015, Trends in Cognitive Sciences.

[29]  David Kemmerer,et al.  Categories of object concepts across languages and brains: the relevance of nominal classification systems to cognitive neuroscience , 2017 .

[30]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[31]  Justin L. Gardner,et al.  Feature-Specific Attentional Priority Signals in Human Cortex , 2011, The Journal of Neuroscience.

[32]  F. Pulvermüller How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics , 2013, Trends in Cognitive Sciences.

[33]  Emiliano Ricciardi,et al.  The blind brain: How (lack of) vision shapes the morphological and functional architecture of the human brain , 2014, Experimental biology and medicine.

[34]  Mario Aguilar,et al.  Predicting Neural Activity Patterns Associated with Sentences Using a Neurobiologically Motivated Model of Semantic Representation , 2016, Cerebral cortex.

[35]  Marco Baroni,et al.  A set of semantic norms for German and Italian , 2011, Behavior research methods.

[36]  Stefania Bracci,et al.  Dissociations and associations between shape and category representations in the two visual pathways. , 2015, Journal of vision.

[37]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[38]  Bradford Z. Mahon,et al.  Spatial Frequency Tuning Reveals Interactions between the Dorsal and Ventral Visual Systems , 2013, Journal of Cognitive Neuroscience.

[39]  A. Treisman,et al.  Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions , 1995, Science.

[40]  Sven J. Dickinson,et al.  Canonical Skeletons for Shape Matching , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[41]  Emiliano Ricciardi,et al.  Foreground Enhancement and Background Suppression in Human Early Visual System During Passive Perception of Natural Images , 2017 .

[42]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[43]  Nikolaus Kriegeskorte,et al.  Unique semantic space in the brain of each beholder predicts perceived similarity , 2014, Proceedings of the National Academy of Sciences.

[44]  Michael F. Bonner,et al.  Converging Evidence for the Neuroanatomic Basis of Combinatorial Semantics in the Angular Gyrus , 2015, The Journal of Neuroscience.

[45]  Alessandro Lenci,et al.  BLIND: a set of semantic feature norms from the congenitally blind , 2013, Behavior research methods.

[46]  S. Cappa,et al.  The neural representation of abstract words: the role of emotion. , 2014, Cerebral cortex.

[47]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[48]  Paul Wright,et al.  Objects and Categories: Feature Statistics and Object Processing in the Ventral Stream , 2013, Journal of Cognitive Neuroscience.

[49]  Philip N. Klein,et al.  Recognition of shapes by editing their shock graphs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Angela D. Friederici,et al.  Pathways to language: fiber tracts in the human brain , 2009, Trends in Cognitive Sciences.

[51]  Mark S. Seidenberg,et al.  Semantic feature production norms for a large set of living and nonliving things , 2005, Behavior research methods.

[52]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[53]  P. Pietrini,et al.  A topographical organization for action representation in the human brain , 2015, Human brain mapping.

[54]  Daria Proklova,et al.  Disentangling Representations of Object Shape and Object Category in Human Visual Cortex: The Animate–Inanimate Distinction , 2016, Journal of Cognitive Neuroscience.

[55]  Alison J. Wiggett,et al.  Behavioral / Systems / Cognitive Functional Magnetic Resonance Imaging Investigation of Overlapping Lateral Occipitotemporal Activations Using Multi-Voxel Pattern Analysis , 2006 .

[56]  M. Seghier The Angular Gyrus , 2013, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[57]  Daniela Perani,et al.  Decoding the neural representation of fine-grained conceptual categories , 2016, NeuroImage.

[58]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[59]  Russell A. Poldrack,et al.  Large-scale automated synthesis of human functional neuroimaging data , 2011, Nature Methods.

[60]  R Cameron Craddock,et al.  A whole brain fMRI atlas generated via spatially constrained spectral clustering , 2012, Human brain mapping.

[61]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[62]  David J. Freedman,et al.  Interaction between Spatial and Feature Attention in Posterior Parietal Cortex , 2016, Neuron.

[63]  Emiliano Ricciardi,et al.  How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge , 2016, NeuroImage.

[64]  Andrew C. Connolly,et al.  Effect of congenital blindness on the semantic representation of some everyday concepts , 2007, Proceedings of the National Academy of Sciences.

[65]  John C Gore,et al.  The role of the parietal cortex in visual feature binding , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Matteo Bianchi,et al.  A synergy-based hand control is encoded in human motor cortical areas , 2016, eLife.

[67]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[68]  Melvyn A. Goodale,et al.  A TMS Investigation on the Role of Lateral Occipital Complex and Caudal Intraparietal Sulcus in the Perception of Object Form and Orientation , 2017, Journal of Cognitive Neuroscience.

[69]  A. Caramazza,et al.  Object Domain and Modality in the Ventral Visual Pathway , 2016, Trends in Cognitive Sciences.

[70]  Sabine Kastner,et al.  The representation of tool and non-tool object information in the human intraparietal sulcus. , 2013, Journal of neurophysiology.

[71]  Timothy J. Andrews,et al.  Spatial properties of objects predict patterns of neural response in the ventral visual pathway , 2016, NeuroImage.

[72]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[73]  Marius V Peelen,et al.  Shape-independent object category responses revealed by MEG and fMRI decoding. , 2016, Journal of neurophysiology.

[74]  Katharina N. Seidl-Rathkopf,et al.  Functions of the human frontoparietal attention network: Evidence from neuroimaging , 2015, Current Opinion in Behavioral Sciences.

[75]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[76]  S. Yantis,et al.  Cortical mechanisms of feature-based attentional control. , 2003, Cerebral cortex.

[77]  Matteo Visconti di Oleggio Castello,et al.  Attention selectively reshapes the geometry of distributed semantic representation , 2016, bioRxiv.

[78]  Tom Hartley,et al.  Low-Level Image Properties of Visual Objects Predict Patterns of Neural Response across Category-Selective Regions of the Ventral Visual Pathway , 2014, The Journal of Neuroscience.

[79]  Bradford Z. Mahon,et al.  What drives the organization of object knowledge in the brain? , 2011, Trends in Cognitive Sciences.

[80]  Bradford Z. Mahon,et al.  Concepts and categories: a cognitive neuropsychological perspective. , 2009, Annual review of psychology.

[81]  Simon B. Eickhoff,et al.  Assignment of functional activations to probabilistic cytoarchitectonic areas revisited , 2007, NeuroImage.

[82]  Li Fei-Fei,et al.  Evidence for similar patterns of neural activity elicited by picture- and word-based representations of natural scenes , 2017, NeuroImage.

[83]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[84]  Cathy J. Price,et al.  A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading , 2012, NeuroImage.

[85]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[86]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[87]  A. Caramazza,et al.  How Visual Is the Visual Cortex? Comparing Connectional and Functional Fingerprints between Congenitally Blind and Sighted Individuals , 2015, The Journal of Neuroscience.

[88]  Franziska R. Richter,et al.  Multimodal Feature Integration in the Angular Gyrus during Episodic and Semantic Retrieval , 2016, The Journal of Neuroscience.

[89]  P. Pietrini,et al.  New light from the dark: what blindness can teach us about brain function. , 2011, Current opinion in neurology.

[90]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[91]  Mark S. Seidenberg,et al.  Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics. , 2016, Cerebral cortex.

[92]  Johan Wagemans,et al.  A conceptual framework of computations in mid-level vision , 2014, Front. Comput. Neurosci..