Beyond the word and image: II- Structural and functional connectivity of a common semantic system

&NA; Understanding events requires interplaying cognitive processes arising in neural networks whose organisation and connectivity remain subjects of controversy in humans. In the present study, by combining diffusion tensor imaging and functional interaction analysis, we aim to provide new insights on the organisation of the structural and functional pathways connecting the multiple nodes of the identified semantic system ‐shared by vision and language (Jouen et al., 2015). We investigated a group of 19 healthy human subjects during experimental tasks of reading sentences or seeing pictures. The structural connectivity was realised by deterministic tractography using an algorithm to extract white matter fibers terminating in the selected regions of interest (ROIs) and the functional connectivity by independent component analysis to measure correlated activities among these ROIs. The major connections link ventral neural stuctures including the parietal and temporal cortices through inferior and middle longitudinal fasciculi, the retrosplenial and parahippocampal cortices through the cingulate bundle, and the temporal and prefrontal structures through the uncinate fasciculus. The imageability score provided when the subject was reading a sentence was significantly correlated with the factor of anisotropy of the left parieto‐temporal connections of the middle longitudinal fasciculus. A large part of this ventrally localised structural connectivity corresponds to functional interactions between the main parietal, temporal and frontal nodes. More precisely, the strong coactivation both in the anterior temporal pole and in the region of the temporo‐parietal cortex suggests dual and cooperating roles for these areas within the semantic system. These findings are discussed in terms of two semantics‐related sub‐systems responsible for conceptual representation. HighlightsStructural and functional connectivity for language and visual scene comprehension.Structural tracks converge‐diverge to the semantic anterior temporal site.Parieto‐temporal cortex is an integrative node between lateral and medial networks.mLF integrity correlates with behavioural comprehension metric.Neural activity is correlated in lateral and in medial nodes.

[1]  J. Hodges,et al.  Non-verbal semantic impairment in semantic dementia , 2000, Neuropsychologia.

[2]  E. Jefferies.  The neural basis of semantic cognition: Converging evidence from neuropsychology, neuroimaging and TMS , 2013, Cortex.

[3]  Mark S. Seidenberg,et al.  Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics. , 2016, Cerebral cortex.

[4]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[5]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[6]  Yong He,et al.  The semantic anatomical network: Evidence from healthy and brain‐damaged patient populations , 2015, Human brain mapping.

[7]  Hugues Duffau,et al.  Anatomic dissection of the inferior fronto-occipital fasciculus revisited in the lights of brain stimulation data , 2010, Cortex.

[8]  Paul Hoffman,et al.  The Semantic Network at Work and Rest: Differential Connectivity of Anterior Temporal Lobe Subregions , 2016, The Journal of Neuroscience.

[9]  A. Caramazza,et al.  White matter structural connectivity underlying semantic processing: evidence from brain damaged patients. , 2013, Brain : a journal of neurology.

[10]  M. Catani,et al.  A diffusion tensor imaging tractography atlas for virtual in vivo dissections , 2008, Cortex.

[11]  Matthew A. Lambon Ralph,et al.  Lateralization of ventral and dorsal auditory-language pathways in the human brain , 2005, NeuroImage.

[12]  P. Paz-Alonso,et al.  Combinatorial semantics strengthens angular-anterior temporal coupling , 2015, Cortex.

[13]  R. Wise,et al.  Semantic retrieval during overt picture description: Left anterior temporal or the parietal lobe? , 2015, Neuropsychologia.

[14]  Lauren L. Cloutman,et al.  The structural connectivity of higher order association cortices reflects human functional brain networks , 2017, Cortex.

[15]  Matthew A. Lambon Ralph,et al.  Differential Contributions of Bilateral Ventral Anterior Temporal Lobe and Left Anterior Superior Temporal Gyrus to Semantic Processes , 2011, Journal of Cognitive Neuroscience.

[16]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[17]  Jordan Henry Grafman,et al.  Impairment of prosocial sentiments is associated with frontopolar and septal damage in frontotemporal dementia , 2011, NeuroImage.

[18]  H. Duffau,et al.  Is the human left middle longitudinal fascicle essential for language? A brain electrostimulation study , 2011, Human brain mapping.

[19]  Elizabeth Jefferies,et al.  Elucidating the Nature of Deregulated Semantic Cognition in Semantic Aphasia: Evidence for the Roles of Prefrontal and Temporo-parietal Cortices , 2010, Journal of Cognitive Neuroscience.

[20]  Volkmar Glauche,et al.  Ventral and dorsal pathways for language , 2008, Proceedings of the National Academy of Sciences.

[21]  Elizabeth Jefferies,et al.  Semantic Processing in the Anterior Temporal Lobes: A Meta-analysis of the Functional Neuroimaging Literature , 2010, Journal of Cognitive Neuroscience.

[22]  Matthew A. Lambon Ralph,et al.  Convergent Connectivity and Graded Specialization in the Rostral Human Temporal Lobe as Revealed by Diffusion-Weighted Imaging Probabilistic Tractography , 2012, Journal of Cognitive Neuroscience.

[23]  Dean Mobbs,et al.  Deconstructing the brain’s moral network: dissociable functionality between the temporoparietal junction and ventro-medial prefrontal cortex , 2013, Social cognitive and affective neuroscience.

[24]  Bruce Crosson,et al.  Neural Basis of Semantic Memory: Role of the basal ganglia in language and semantics: supporting cast , 2007 .

[25]  Heidi Johansen-Berg,et al.  White matter integrity in the vicinity of Broca's area predicts grammar learning success , 2009, NeuroImage.

[26]  Torsten Rohlfing,et al.  Problem Solving, Working Memory, and Motor Correlates of Association and Commissural Fiber Bundles in Normal Aging: a Quantitative Fiber Tracking Study , 2022 .

[27]  Denise Y. Harvey,et al.  Neuropsychological evidence for the functional role of the uncinate fasciculus in semantic control , 2012, Neuropsychologia.

[28]  M. L. Lambon Ralph,et al.  Semantic impairment in stroke aphasia versus semantic dementia: a case-series comparison. , 2006, Brain : a journal of neurology.

[29]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[30]  B. Miller,et al.  Anterior temporal lobe degeneration produces widespread network-driven dysfunction. , 2013, Brain : a journal of neurology.

[31]  Jessica A. Turner,et al.  Behavioral Interpretations of Intrinsic Connectivity Networks , 2011, Journal of Cognitive Neuroscience.

[32]  T. Rogers,et al.  Lichtheim 2: Synthesizing Aphasia and the Neural Basis of Language in a Neurocomputational Model of the Dual Dorsal-Ventral Language Pathways , 2011, Neuron.

[33]  Richard S. J. Frackowiak,et al.  Functional anatomy of a common semantic system for words and pictures , 1996, Nature.

[34]  A. Damasio,et al.  Convergence and divergence in a neural architecture for recognition and memory , 2009, Trends in Neurosciences.

[35]  R. Ilmoniemi,et al.  Functional links between motor and language systems , 2005, The European journal of neuroscience.

[36]  A. Caramazza,et al.  Predicting Conceptual Processing Capacity from Spontaneous Neuronal Activity of the Left Middle Temporal Gyrus , 2012, The Journal of Neuroscience.

[37]  T. Rogers,et al.  The neural and computational bases of semantic cognition , 2016, Nature Reviews Neuroscience.

[38]  P. A. Lewis,et al.  Brain mechanisms for mood congruent memory facilitation , 2005, NeuroImage.

[39]  Stefan Klöppel,et al.  Combining Functional and Anatomical Connectivity Reveals Brain Networks for Auditory Language Comprehension , 2022 .

[40]  Habib Benali,et al.  Regions, systems, and the brain: Hierarchical measures of functional integration in fMRI , 2008, Medical Image Anal..

[41]  T. Rogers,et al.  A unified model of human semantic knowledge and its disorders , 2017, Nature Human Behaviour.

[42]  Kai Spiegelhalder,et al.  Objective sleep disturbances are associated with greater waking resting-state connectivity between the retrosplenial cortex/ hippocampus and various nodes of the default mode network. , 2016, Journal of psychiatry & neuroscience : JPN.

[43]  Hugues Duffau,et al.  Mapping the connectivity underlying multimodal (verbal and non-verbal) semantic processing: A brain electrostimulation study , 2013, Neuropsychologia.

[44]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[45]  Hugues Duffau,et al.  Integrating emotional valence and semantics in the human ventral stream: a hodological account , 2015, Front. Psychol..

[46]  Hugues Duffau,et al.  Toward a pluri-component, multimodal, and dynamic organization of the ventral semantic stream in humans: lessons from stimulation mapping in awake patients , 2013, Front. Syst. Neurosci..

[47]  P. Grenier,et al.  MR imaging of intravoxel incoherent motions: application to diffusion and perfusion in neurologic disorders. , 1986, Radiology.

[48]  Richard J. Binney,et al.  Differing contributions of inferior prefrontal and anterior temporal cortex to concrete and abstract conceptual knowledge , 2015, Cortex.

[49]  Rutvik H. Desai,et al.  The neurobiology of semantic memory , 2011, Trends in Cognitive Sciences.

[50]  D. Poeppel,et al.  Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language , 2004, Cognition.

[51]  Mark S. Seidenberg,et al.  Heteromodal Cortical Areas Encode Sensory-Motor Features of Word Meaning , 2016, The Journal of Neuroscience.

[52]  Matthew A. Lambon Ralph,et al.  Neurocognitive insights on conceptual knowledge and its breakdown , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[53]  B. MacWhinney.  A UNIFIED MODEL , 2007 .

[54]  Habib Benali,et al.  Dynamics of motor-related functional integration during motor sequence learning , 2010, NeuroImage.

[55]  Michael F. Bonner,et al.  Converging Evidence for the Neuroanatomic Basis of Combinatorial Semantics in the Angular Gyrus , 2015, The Journal of Neuroscience.

[56]  劉 健勤,et al.  複雑系に見たDefault Mode Network , 2009, ICS 2009.

[57]  Matthew F Glasser,et al.  DTI tractography of the human brain's language pathways. , 2008, Cerebral cortex.

[58]  Alexander Leemans,et al.  The B‐matrix must be rotated when correcting for subject motion in DTI data , 2009, Magnetic resonance in medicine.

[59]  L. Barsalou.  Grounded cognition. , 2008, Annual review of psychology.

[60]  Guillaume Marrelec,et al.  Contribution of Exploratory Methods to the Investigation of Extended Large-Scale Brain Networks in Functional MRI: Methodologies, Results, and Challenges , 2008, Int. J. Biomed. Imaging.

[61]  Rutvik H. Desai,et al.  Toward a brain-based componential semantic representation , 2016, Cognitive neuropsychology.

[62]  S. Thompson-Schill.  Neuroimaging studies of semantic memory: inferring “how” from “where” , 2003, Neuropsychologia.

[63]  Istvan Molnar-Szakacs,et al.  Self-Processing and the Default Mode Network: Interactions with the Mirror Neuron System , 2013, Front. Hum. Neurosci..

[64]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[65]  A. Damasio.  Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition , 1989, Cognition.

[66]  Jean-Philippe Thiran,et al.  Hand preference and sex shape the architecture of language networks , 2006, Human brain mapping.

[67]  Simon B Eickhoff,et al.  Investigating the Functional Heterogeneity of the Default Mode Network Using Coordinate-Based Meta-Analytic Modeling , 2009, The Journal of Neuroscience.

[68]  G. Shulman,et al.  Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[69]  Bradford Z. Mahon,et al.  Arguments about the nature of concepts: Symbols, embodiment, and beyond , 2016, Psychonomic bulletin & review.

[70]  Peter Ford Dominey,et al.  Beyond the word and image: characteristics of a common meaning system for language and vision revealed by functional and structural imaging , 2015, NeuroImage.

[71]  D. Schacter,et al.  The Brain's Default Network , 2008, Annals of the New York Academy of Sciences.

[72]  Georg Northoff,et al.  Self-referential processing in our brain—A meta-analysis of imaging studies on the self , 2006, NeuroImage.

[73]  Peter Ford Dominey,et al.  Sequential Coherence in Sentence Pairs Enhances Imagery during Comprehension: An Individual Differences Study , 2015, PloS one.

[74]  A. Bailey,et al.  Are there theory of mind regions in the brain? A review of the neuroimaging literature , 2009, Human brain mapping.

[75]  Timothy M. Ellmore,et al.  Temporal lobe white matter asymmetry and language laterality in epilepsy patients , 2010, NeuroImage.

[76]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[77]  G Marrelec,et al.  NetBrainWork: a toolbox for studying functional interactions in large-scale brain networks in fMRI. , 2009, NeuroImage.

[78]  Hugues Duffau,et al.  The left inferior fronto-occipital fasciculus subserves language semantics: a multilevel lesion study , 2014, Brain Structure and Function.

[79]  E. Jefferies,et al.  Amodal semantic representations depend on both anterior temporal lobes: Evidence from repetitive transcranial magnetic stimulation , 2010, Neuropsychologia.

[80]  Philip A. Cook,et al.  Heteromodal conceptual processing in the angular gyrus , 2013, NeuroImage.

[81]  Peter Fransson,et al.  The precuneus/posterior cingulate cortex plays a pivotal role in the default mode network: Evidence from a partial correlation network analysis , 2008, NeuroImage.

[82]  Derek K. Jones,et al.  Perisylvian language networks of the human brain , 2005, Annals of neurology.

[83]  Ling Li,et al.  The neural basis of semantic cognition in Mandarin Chinese: A combined fMRI and TMS study , 2019, Human brain mapping.

[84]  P. Basser,et al.  In vivo fiber tractography using DT‐MRI data , 2000, Magnetic resonance in medicine.

[85]  E. Maguire,et al.  What does the retrosplenial cortex do? , 2009, Nature Reviews Neuroscience.

[86]  John S. Duncan,et al.  Hemispheric asymmetries in language-related pathways: A combined functional MRI and tractography study , 2006, NeuroImage.

[87]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[88]  Penelope A. Lewis,et al.  Ventromedial prefrontal volume predicts understanding of others and social network size , 2011, NeuroImage.

[89]  Nikos Makris,et al.  Mapping temporo-parietal and temporo-occipital cortico-cortical connections of the human middle longitudinal fascicle in subject-specific, probabilistic, and stereotaxic Talairach spaces , 2016, Brain Imaging and Behavior.

[90]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[91]  Wanqing Li,et al.  The default mode network and social understanding of others: what do brain connectivity studies tell us , 2014, Front. Hum. Neurosci..

[92]  Brian A. Wandell,et al.  Exploring connectivity of the brain's white matter with dynamic queries , 2005, IEEE Transactions on Visualization and Computer Graphics.

[93]  P. Hoffman,et al.  The Roles of Left Versus Right Anterior Temporal Lobes in Conceptual Knowledge: An ALE Meta-analysis of 97 Functional Neuroimaging Studies , 2015, Cerebral cortex.

[94]  Hanna Damasio,et al.  Neural convergence and divergence in the mammalian cerebral cortex: From experimental neuroanatomy to functional neuroimaging , 2013, The Journal of comparative neurology.

[95]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[96]  D. Pandya,et al.  The extreme capsule in humans and rethinking of the language circuitry , 2009, Brain Structure and Function.

[97]  D. Pandya,et al.  Delineation of the middle longitudinal fascicle in humans: a quantitative, in vivo, DT-MRI study. , 2009, Cerebral cortex.

[98]  Hugues Duffau,et al.  Intraoperative cortico–subcortical stimulations in surgery of low-grade gliomas , 2005, Expert review of neurotherapeutics.

[99]  Habib Benali,et al.  NEDICA: Detection of group functional networks in FMRI using spatial independent component analysis , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.