Knowing what you need to know in advance: The neural processes underpinning flexible semantic retrieval of thematic and taxonomic relations

Semantic retrieval is flexible, allowing us to focus on subsets of features and associations that are relevant to the current task or context: for example, we use taxonomic relations to locate items in the supermarket (carrots are a vegetable), but thematic associations to decide which tools we need when cooking (carrot goes with peeler). We used fMRI to investigate the neural basis of this form of semantic flexibility; in particular, we asked how retrieval unfolds differently when participants have advanced knowledge of the type of link to retrieve between concepts (taxonomic or thematic). Participants performed a semantic relatedness judgement task: on half the trials, they were cued to search for a taxonomic or thematic link, while on the remaining trials, they judged relatedness without knowing which type of semantic relationship would be relevant. Left inferior frontal gyrus showed greater activation when participants knew the trial type in advance. An overlapping region showed a stronger response when the semantic relationship between the items was weaker, suggesting this structure supports both top-down and bottom-up forms of semantic control. Multivariate pattern analysis further revealed that the neural response in left inferior frontal gyrus reflects goal information related to different conceptual relationships. Top-down control specifically modulated the response in visual cortex: when the goal was unknown, there was greater deactivation to the first word, and greater activation to the second word. We conclude that top-down control of semantic retrieval is primarily achieved through the gating of task-relevant ‘spoke’ regions.

[1]  T. Allison,et al.  Word recognition in the human inferior temporal lobe , 1994, Nature.

[2]  Marisa O. Hollinshead,et al.  The organization of the human cerebral cortex estimated by intrinsic functional connectivity. , 2011, Journal of neurophysiology.

[3]  D. Swinney Lexical access during sentence comprehension: (Re)consideration of context effects , 1979 .

[4]  R. Poldrack,et al.  Dissociable Controlled Retrieval and Generalized Selection Mechanisms in Ventrolateral Prefrontal Cortex , 2005, Neuron.

[5]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[6]  Daniel Mirman,et al.  The cost of switching between taxonomic and thematic semantics , 2018, Memory & cognition.

[7]  Paul Hoffman,et al.  An individual differences approach to semantic cognition: Divergent effects of age on representation, retrieval and selection , 2017, bioRxiv.

[8]  Z. Estes,et al.  Thematic thinking : the apprehension and consequences of thematic relations , 2011 .

[9]  Stefan Pollmann,et al.  PyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI Data , 2009, Neuroinformatics.

[10]  L. Buxbaum,et al.  Visual context modulates potentiation of grasp types during semantic object categorization , 2013, Psychonomic Bulletin & Review.

[11]  Kaustubh Supekar,et al.  Dissociable connectivity within human angular gyrus and intraparietal sulcus: evidence from functional and structural connectivity. , 2010, Cerebral cortex.

[12]  Gina F. Humphreys,et al.  Controlled semantic cognition relies upon dynamic and flexible interactions between the executive ‘semantic control’ and hub-and-spoke ‘semantic representation’ systems , 2018, Cortex.

[13]  M. Denis,et al.  Cortical anatomy of mental imagery of concrete nouns based on their dictionary definition , 1998, Neuroreport.

[14]  Lars Muckli,et al.  Decoding Sound and Imagery Content in Early Visual Cortex , 2014, Current Biology.

[15]  Elizabeth Jefferies,et al.  Fractionating the anterior temporal lobe: MVPA reveals differential responses to input and conceptual modality , 2017, NeuroImage.

[16]  Paul Wright,et al.  Objects and Categories: Feature Statistics and Object Processing in the Ventral Stream , 2013, Journal of Cognitive Neuroscience.

[17]  Elizabeth Jefferies,et al.  Shared processes resolve competition within and between episodic and semantic memory: Evidence from patients with LIFG lesions , 2018, Cortex.

[18]  P. Fletcher,et al.  Selecting among competing alternatives: selection and retrieval in the left inferior frontal gyrus. , 2005, Cerebral cortex.

[19]  Mark W. Woolrich,et al.  Robust group analysis using outlier inference , 2008, NeuroImage.

[20]  Stephen M. Smith,et al.  Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data , 2001, NeuroImage.

[21]  J. Smallwood,et al.  Automatic and Controlled Semantic Retrieval: TMS Reveals Distinct Contributions of Posterior Middle Temporal Gyrus and Angular Gyrus , 2015, The Journal of Neuroscience.

[22]  Elizabeth Jefferies,et al.  Executive Semantic Processing Is Underpinned by a Large-scale Neural Network: Revealing the Contribution of Left Prefrontal, Posterior Temporal, and Parietal Cortex to Controlled Retrieval and Selection Using TMS , 2012, Journal of Cognitive Neuroscience.

[23]  Karin Ackermann,et al.  Categories and Concepts , 2003, Job 28. Cognition in Context.

[24]  Thomas T. Liu,et al.  A component based noise correction method (CompCor) for BOLD and perfusion based fMRI , 2007, NeuroImage.

[25]  Regine Zopf,et al.  Multisensory coding in the multiple-demand regions: vibrotactile task information is coded in frontoparietal cortex. , 2017, Journal of neurophysiology.

[26]  Elizabeth Jefferies,et al.  Dissociations in semantic cognition: Oscillatory evidence for opposing effects of semantic control and type of semantic relation in anterior and posterior temporal cortex , 2019, Cortex.

[27]  D. Tranel,et al.  Damage to the default mode network disrupts autobiographical memory retrieval. , 2015, Social cognitive and affective neuroscience.

[28]  Francisco Pereira,et al.  A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data , 2016, Cognitive neuropsychology.

[29]  S. Thompson-Schill,et al.  Manual Experience Shapes Object Representations , 2013, Psychological science.

[30]  Susan L. Whitfield-Gabrieli,et al.  Conn: A Functional Connectivity Toolbox for Correlated and Anticorrelated Brain Networks , 2012, Brain Connect..

[31]  Troy A. Smith,et al.  Human hippocampus represents space and time during retrieval of real-world memories , 2015, Proceedings of the National Academy of Sciences.

[32]  Uta Noppeney,et al.  Retrieval of abstract semantics , 2004, NeuroImage.

[33]  Gina F. Humphreys,et al.  Fusion and Fission of Cognitive Functions in the Human Parietal Cortex , 2014, Cerebral cortex.

[34]  T. Schnur,et al.  Role of features and categories in the organization of object knowledge: Evidence from adaptation fMRI , 2016, Cortex.

[35]  K. McGregor,et al.  Conceptual organization at 6 and 8 years of age: evidence from the semantic priming of object decisions. , 2007, Journal of speech, language, and hearing research : JSLHR.

[36]  Morten L. Kringelbach,et al.  Visual word recognition: the first half second , 2004, NeuroImage.

[37]  S. Krach,et al.  Taxonomic and thematic categories: Neural correlates of categorization in an auditory-to-visual priming task using fMRI , 2009, Brain Research.

[38]  A. Wagner,et al.  Domain-general and domain-sensitive prefrontal mechanisms for recollecting events and detecting novelty. , 2005, Cerebral cortex.

[39]  Daniel J Mitchell,et al.  Task Encoding across the Multiple Demand Cortex Is Consistent with a Frontoparietal and Cingulo-Opercular Dual Networks Distinction , 2016, The Journal of Neuroscience.

[40]  Robin K. Morris,et al.  Lexical Processing and Sentence Context Effects , 2006 .

[41]  Alissa Melinger,et al.  The dynamic microstructure of speech production: semantic interference built on the fly. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[42]  A. Caramazza,et al.  Conceptual Object Representations in Human Anterior Temporal Cortex , 2012, The Journal of Neuroscience.

[43]  E. Maguire,et al.  The Human Hippocampus and Spatial and Episodic Memory , 2002, Neuron.

[44]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[45]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[46]  Sharon L. Thompson-Schill,et al.  Color, Context, and Cognitive Style: Variations in Color Knowledge Retrieval as a Function of Task and Subject Variables , 2011, Journal of Cognitive Neuroscience.

[47]  Daniel S. Margulies,et al.  NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain , 2014, bioRxiv.

[48]  Alexis Amadon,et al.  Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding , 2016, NeuroImage.

[49]  L. Cipolotti,et al.  Dynamic aphasia: an inability to select between competing verbal responses? , 1998, Brain : a journal of neurology.

[50]  Matthew H. Davis,et al.  Dissociating frontotemporal contributions to semantic ambiguity resolution in spoken sentences. , 2012, Cerebral cortex.

[51]  Elizabeth Jefferies,et al.  Semantic control deficits impair understanding of thematic relationships more than object identity , 2017, Neuropsychologia.

[52]  Nancy Kanwisher,et al.  Broad domain generality in focal regions of frontal and parietal cortex , 2013, Proceedings of the National Academy of Sciences.

[53]  Wolf Singer,et al.  Smaller Primary Visual Cortex Is Associated with Stronger, but Less Precise Mental Imagery. , 2016, Cerebral cortex.

[54]  Elizabeth Jefferies,et al.  Imagining Sounds and Images: Decoding the Contribution of Unimodal and Transmodal Brain Regions to Semantic Retrieval in the Absence of Meaningful Input , 2019, Journal of Cognitive Neuroscience.

[55]  R. H. Baayen,et al.  The CELEX Lexical Database (CD-ROM) , 1996 .

[56]  Elizabeth Jefferies,et al.  Exploring the role of the posterior middle temporal gyrus in semantic cognition: Integration of anterior temporal lobe with executive processes , 2016, NeuroImage.

[57]  Elizabeth Jefferies,et al.  Semantic Processing in the Anterior Temporal Lobes: A Meta-analysis of the Functional Neuroimaging Literature , 2010, Journal of Cognitive Neuroscience.

[58]  Elizabeth Jefferies,et al.  Both the Middle Temporal Gyrus and the Ventral Anterior Temporal Area Are Crucial for Multimodal Semantic Processing: Distortion-corrected fMRI Evidence for a Double Gradient of Information Convergence in the Temporal Lobes , 2012, Journal of Cognitive Neuroscience.

[59]  D. Margulies,et al.  Default mode network can support the level of detail in experience during active task states , 2018, Proceedings of the National Academy of Sciences.

[60]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[61]  Roy W Jones,et al.  The degraded concept representation system in semantic dementia: damage to pan-modal hub, then visual spoke. , 2012, Brain : a journal of neurology.

[62]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[63]  Irene P. Kan,et al.  ROLE OF MENTAL IMAGERY IN A PROPERTY VERIFICATION TASK: FMRI EVIDENCE FOR PERCEPTUAL REPRESENTATIONS OF CONCEPTUAL KNOWLEDGE , 2003, Cognitive neuropsychology.

[64]  Elizabeth Jefferies,et al.  Down but not out in posterior cingulate cortex: Deactivation yet functional coupling with prefrontal cortex during demanding semantic cognition , 2016, NeuroImage.

[65]  Johannes Jacobus Fahrenfort,et al.  Current and future goals are represented in opposite patterns in object-selective cortex , 2018, bioRxiv.

[66]  C. Davis N-Watch: A program for deriving neighborhood size and other psycholinguistic statistics , 2005, Behavior research methods.

[67]  M. L. Lambon Ralph,et al.  The Neural Organization of Semantic Control: TMS Evidence for a Distributed Network in Left Inferior Frontal and Posterior Middle Temporal Gyrus , 2010, Cerebral cortex.

[68]  Yi Zeng,et al.  Representational similarity analysis reveals task-dependent semantic influence of the visual word form area , 2018, Scientific Reports.

[69]  R. Nathan Spreng,et al.  The Common Neural Basis of Autobiographical Memory, Prospection, Navigation, Theory of Mind, and the Default Mode: A Quantitative Meta-analysis , 2009, Journal of Cognitive Neuroscience.

[70]  Matthew A. Lambon Ralph,et al.  Unveiling the dynamic interplay between the hub- and spoke-components of the brain's semantic system and its impact on human behaviour , 2019, NeuroImage.

[71]  J. Duncan,et al.  Adaptive Coding of Task-Relevant Information in Human Frontoparietal Cortex , 2011, The Journal of Neuroscience.

[72]  W. Marslen-Wilson,et al.  Access to word meanings during spoken language comprehension: effects of sentential semantic context. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[73]  Katie L McMahon,et al.  Differential processing of thematic and categorical conceptual relations in spoken word production. , 2013, Journal of experimental psychology. General.

[74]  Kevin Murphy,et al.  The impact of global signal regression on resting state correlations: Are anti-correlated networks introduced? , 2009, NeuroImage.

[75]  Blair C. Armstrong,et al.  The what, when, where, and how of visual word recognition , 2014, Trends in Cognitive Sciences.

[76]  Gregory L. Murphy,et al.  Thematic relations in adults' concepts. , 2001, Journal of experimental psychology. General.

[77]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[78]  Kurt E. Weaver,et al.  Mapping anterior temporal lobe language areas with fMRI: A multicenter normative study , 2011, NeuroImage.

[79]  T. Rogers,et al.  The neural and computational bases of semantic cognition , 2016, Nature Reviews Neuroscience.

[80]  Elizabeth Jefferies,et al.  Conceptual control across modalities: graded specialisation for pictures and words in inferior frontal and posterior temporal cortex , 2015, Neuropsychologia.

[81]  Elizabeth Jefferies,et al.  Situating the default-mode network along a principal gradient of macroscale cortical organization , 2016, Proceedings of the National Academy of Sciences.

[82]  Denise Y. Harvey,et al.  Neuropsychological evidence for the functional role of the uncinate fasciculus in semantic control , 2012, Neuropsychologia.

[83]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[84]  T. Rogers,et al.  Anterior temporal cortex and semantic memory: Reconciling findings from neuropsychology and functional imaging , 2006, Cognitive, affective & behavioral neuroscience.

[85]  Lee Ryan,et al.  Hippocampal activation during episodic and semantic memory retrieval: Comparing category production and category cued recall , 2008, Neuropsychologia.

[86]  Karl J. Friston,et al.  Distinct Top-down and Bottom-up Brain Connectivity During Visual Perception and Imagery , 2017, Scientific Reports.

[87]  J. Hampton Concepts as Prototypes , 2006 .

[88]  W. Levelt,et al.  Effects of semantic context in the naming of pictures and words , 2001, Cognition.

[89]  J. Rodd,et al.  Anteromedial temporal cortex supports fine-grained differentiation among objects. , 2005, Cerebral cortex.

[90]  A. Bartels,et al.  Decoding the Yellow of a Gray Banana , 2013, Current Biology.

[91]  W. Schneider,et al.  Perceptual Knowledge Retrieval Activates Sensory Brain Regions , 2006, The Journal of Neuroscience.

[92]  Rebecca L. Jackson,et al.  The Nature and Neural Correlates of Semantic Association versus Conceptual Similarity , 2015, Cerebral cortex.

[93]  Xuchu Weng,et al.  Semantic processing of Chinese in left inferior prefrontal cortex studied with reversible words , 2004, NeuroImage.

[94]  Elizabeth Jefferies,et al.  Charting the effects of TMS with fMRI: Modulation of cortical recruitment within the distributed network supporting semantic control , 2016, Neuropsychologia.

[95]  A. Turken,et al.  The Neural Architecture of the Language Comprehension Network: Converging Evidence from Lesion and Connectivity Analyses , 2011, Front. Syst. Neurosci..

[96]  Donna Rose Addis,et al.  Prefrontal and hippocampal contributions to the generation and binding of semantic associations during successful encoding , 2006, NeuroImage.

[97]  David Badre,et al.  Left ventrolateral prefrontal cortex and the cognitive control of memory , 2007, Neuropsychologia.

[98]  E. Jefferies The neural basis of semantic cognition: Converging evidence from neuropsychology, neuroimaging and TMS , 2013, Cortex.

[99]  Grant M. Walker,et al.  Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain , 2011, Proceedings of the National Academy of Sciences.

[100]  Timothy T. Rogers,et al.  Reverse-Engineering the Cortical Architecture for Controlled Semantic Cognition , 2019, Nature Human Behaviour.

[101]  Hao-Ting Wang,et al.  Varieties of semantic cognition revealed through simultaneous decomposition of intrinsic brain connectivity and behaviour , 2017, NeuroImage.

[102]  M. Farah,et al.  Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[103]  A. Ellis,et al.  Oscillatory Dynamics Supporting Semantic Cognition: MEG Evidence for the Contribution of the Anterior Temporal Lobe Hub and Modality-Specific Spokes , 2017, PloS one.

[104]  R. Kreuz,et al.  Context can constrain lexical access: implications for models of language comprehension , 1986 .

[105]  G. Murphy,et al.  Thematic relations in adults' concepts. , 2001, Journal of experimental psychology. General.

[106]  Paul Hoffman,et al.  Concrete versus abstract forms of social concept: an fMRI comparison of knowledge about people versus social terms , 2018, Philosophical Transactions of the Royal Society B: Biological Sciences.

[107]  Qiong Zhang,et al.  Semantic knowledge influences whether novel episodic associations are represented symmetrically or asymmetrically , 2019, Memory & Cognition.

[108]  Monica Baciu,et al.  The sensory-motor specificity of taxonomic and thematic conceptual relations: A behavioral and fMRI study , 2009, NeuroImage.

[109]  Jemett L. Desmond,et al.  Semantic encoding and retrieval in the left inferior prefrontal cortex: a functional MRI study of task difficulty and process specificity , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[110]  Daniel S. Margulies,et al.  Distinct patterns of thought mediate the link between brain functional connectome and psychological well-being , 2019, bioRxiv.

[111]  Elizabeth Jefferies,et al.  Elucidating the Nature of Deregulated Semantic Cognition in Semantic Aphasia: Evidence for the Roles of Prefrontal and Temporo-parietal Cortices , 2010, Journal of Cognitive Neuroscience.

[112]  Lee Ryan,et al.  Hippocampal activation during retrieval of spatial context from episodic and semantic memory , 2010, Behavioural Brain Research.

[113]  Antony B. Morland,et al.  On the Role of Suppression in Spatial Attention: Evidence from Negative BOLD in Human Subcortical and Cortical Structures , 2014, The Journal of Neuroscience.

[114]  R. Buckner,et al.  Common Prefrontal Regions Coactivate with Dissociable Posterior Regions during Controlled Semantic and Phonological Tasks , 2002, Neuron.

[115]  R. Poldrack,et al.  Recovering Meaning Left Prefrontal Cortex Guides Controlled Semantic Retrieval , 2001, Neuron.

[116]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[117]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[118]  F. Keil,et al.  Early differentiation of causal mechanisms appropriate to biological and nonbiological kinds. , 1991, Child development.

[119]  Elizabeth Jefferies,et al.  Emotion and location cues bias conceptual retrieval in people 1 with deficient semantic control 2 , 2019 .

[120]  P. Hoffman,et al.  Ventrolateral Prefrontal Cortex Plays an Executive Regulation Role in Comprehension of Abstract Words: Convergent Neuropsychological and Repetitive TMS Evidence , 2010, The Journal of Neuroscience.

[121]  David Poeppel,et al.  The neural bases of taxonomic and thematic conceptual relations: An MEG study , 2015, Neuropsychologia.

[122]  L. Buxbaum,et al.  Knowledge of object manipulation and object function: dissociations in apraxic and nonapraxic subjects , 2002, Brain and Language.

[123]  M. Farah,et al.  A neural basis for category and modality specificity of semantic knowledge , 1999, Neuropsychologia.

[124]  Hao-Ting Wang,et al.  Patterns of thought: Population variation in the associations between large-scale network organisation and self-reported experiences at rest , 2018, NeuroImage.

[125]  Dost Öngür,et al.  Anticorrelations in resting state networks without global signal regression , 2012, NeuroImage.

[126]  K. Patterson,et al.  Deficits of knowledge vs . executive control in semantic cognition : Insights from cued naming , 2008 .

[127]  Sally A. Linkenauger,et al.  A Functional Role for Motor Simulation in Identifying Tools , 2010, Psychological science.

[128]  Nadim Joni Shah,et al.  Probabilistic fibre tract analysis of cytoarchitectonically defined human inferior parietal lobule areas reveals similarities to macaques , 2011, NeuroImage.

[129]  Elizabeth Jefferies,et al.  Going beyond Inferior Prefrontal Involvement in Semantic Control: Evidence for the Additional Contribution of Dorsal Angular Gyrus and Posterior Middle Temporal Cortex , 2013, Journal of Cognitive Neuroscience.

[130]  L. Tyler,et al.  Object-Specific Semantic Coding in Human Perirhinal Cortex , 2014, The Journal of Neuroscience.

[131]  Micah B. Goldwater,et al.  The empirical case for role-governed categories , 2011, Cognition.

[132]  Richard J. Binney,et al.  Does the sound of a barking dog activate its corresponding visual form? An fMRI investigation of modality-specific semantic access , 2016, Brain and Language.

[133]  P. Hoffman,et al.  The Roles of Left Versus Right Anterior Temporal Lobes in Conceptual Knowledge: An ALE Meta-analysis of 97 Functional Neuroimaging Studies , 2015, Cerebral cortex.