Neuroanatomical distinctions within the semantic system during sentence comprehension: Evidence from functional magnetic resonance imaging

To make sense of a sentence, we must compute morphosyntactic and semantic-thematic relationships between its verbs and arguments and evaluate the resulting propositional meaning against any preceding context and our real-world knowledge. Recent electrophysiological studies suggest that, in comparison with non-violated verbs (e.g. "...at breakfast the boys would eat..."), animacy semantic-thematically violated verbs (e.g. "...at breakfast the eggs would eat...") and morphosyntactically violated verbs (e.g. "...at breakfast the boys would eats...") evoke a similar neural response. This response is distinct from that evoked by verbs that only violate real-world knowledge (e.g. "...at breakfast the boys would plant..."). Here we used fMRI to examine the neuroanatomical regions engaged in response to these three violations. Real-world violations, relative to other sentence types, led to increased activity within the left anterior inferior frontal cortex, reflecting participants' increased and prolonged efforts to retrieve semantic knowledge about the likelihood of events occurring in the real world. In contrast, animacy semantic-thematic violations of the actions depicted by the central verbs engaged a frontal/inferior parietal/basal ganglia network known to mediate the execution and comprehension of goal-directed action. We suggest that the recruitment of this network reflected a semantic-thematic combinatorial process that involved an attempt to determine whether the actions described by the verbs could be executed by their NP Agents. Intriguingly, this network was also activated to morphosyntactic violations between the verbs and their subject NP arguments. Our findings support the pattern of electrophysiological findings in suggesting (a) that a clear division within the semantic system plays out during sentence comprehension, and (b) that semantic-thematic and syntactic violations of verbs within simple active sentences are treated similarly by the brain.

[1]  E. Reed The Ecological Approach to Visual Perception , 1989 .

[2]  D. Caplan,et al.  The role of animacy and thematic relationships in processing active English sentences: Evidence from event-related potentials , 2007, Brain and Language.

[3]  A. Friederici,et al.  The brain basis of syntactic processes: functional imaging and lesion studies , 2003, NeuroImage.

[4]  John C J Hoeks,et al.  Seeing words in context: the interaction of lexical and sentence level information during reading. , 2004, Brain research. Cognitive brain research.

[5]  David Kemmerer,et al.  Title: Action Verbs, Argument Structure Constructions, and the Mirror Neuron System , 2022 .

[6]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[7]  H. Kolk,et al.  Monitoring in language perception: The effect of misspellings of words in highly constrained sentences , 2006, Brain Research.

[8]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[9]  Peter F. Liddle,et al.  Reading Anomalous Sentences: An Event-Related fMRI Study of Semantic Processing , 2002, NeuroImage.

[10]  A M Dale,et al.  Randomized event‐related experimental designs allow for extremely rapid presentation rates using functional MRI , 1998, Neuroreport.

[11]  M. Scherg,et al.  Localizing P300 Generators in Visual Target and Distractor Processing: A Combined Event-Related Potential and Functional Magnetic Resonance Imaging Study , 2004, The Journal of Neuroscience.

[12]  Charles D. Smith,et al.  Dissociation of Automatic and Strategic Lexical-Semantics: Functional Magnetic Resonance Imaging Evidence for Differing Roles of Multiple Frontotemporal Regions , 2006, The Journal of Neuroscience.

[13]  P. Holcomb,et al.  Event-related brain potentials elicited by syntactic anomaly , 1992 .

[14]  E. Donchin,et al.  Is the P300 component a manifestation of context updating? , 1988, Behavioral and Brain Sciences.

[15]  Angela D. Friederici,et al.  Mapping sentence form onto meaning: The syntax–semantic interface , 2007, Brain Research.

[16]  G. Miller,et al.  Linguistic theory and psychological reality , 1982 .

[17]  Anders M. Dale,et al.  N400-like Magnetoencephalography Responses Modulated by Semantic Context, Word Frequency, and Lexical Class in Sentences , 2002, NeuroImage.

[18]  A. A. Wijers,et al.  Activations of “motor” and other non-language structures during sentence comprehension , 2004, Brain and Language.

[19]  L. Stowe,et al.  Rethinking the neurological basis of language , 2005 .

[20]  J. Grose-Fifer,et al.  Priming by natural category membership in the left and right cerebral hemispheres , 2004, Neuropsychologia.

[21]  H. Neville,et al.  An Event-Related fMRI Study of Syntactic and Semantic Violations , 2001, Journal of psycholinguistic research.

[22]  David R. Dowty On the Semantic Content of the Notion of ‘Thematic Role’ , 1989 .

[23]  Barbara H. Partee,et al.  Properties, types and meaning , 1988 .

[24]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[25]  J. Grose-Fifer,et al.  Evidence for a New Conceptualization of Semantic Representation in the Left and Right Cerebral Hemispheres , 2004, Cortex.

[26]  Phillip J. Holcomb,et al.  Two Neurocognitive Mechanisms of Semantic Integration during the Comprehension of Visual Real-world Events , 2008, Journal of Cognitive Neuroscience.

[27]  Colin M. Brown,et al.  The syntactic positive shift (sps) as an erp measure of syntactic processing , 1993 .

[28]  D. Shankweiler,et al.  An Event-related Neuroimaging Study Distinguishing Form and Content in Sentence Processing , 2000, Journal of Cognitive Neuroscience.

[29]  J. Decety,et al.  Distinct Areas in Parietal Cortex Involved in Long-Term and Short-Term Action Planning: A Pet Investigation , 2002, Cortex.

[30]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[31]  D. V. von Cramon,et al.  Syntactic language processing: ERP lesion data on the role of the basal ganglia , 2003, Journal of the International Neuropsychological Society.

[32]  Michael S. Beauchamp,et al.  Automatic Priming of Semantically Related Words Reduces Activity in the Fusiform Gyrus , 2005, Journal of Cognitive Neuroscience.

[33]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[34]  G. Rizzolatti,et al.  Understanding motor events: a neurophysiological study , 2004, Experimental Brain Research.

[35]  Gina R Kuperberg,et al.  Making sense of sentences in schizophrenia: electrophysiological evidence for abnormal interactions between semantic and syntactic processing. , 2006, Journal of abnormal psychology.

[36]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[37]  M. R. Manzini Learnability and Cognition , 1991 .

[38]  M. Arbib From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics , 2005, Behavioral and Brain Sciences.

[39]  D. Collins,et al.  Automatic 3D Intersubject Registration of MR Volumetric Data in Standardized Talairach Space , 1994, Journal of computer assisted tomography.

[40]  G. Rizzolatti,et al.  Parietal Lobe: From Action Organization to Intention Understanding , 2005, Science.

[41]  W Ni,et al.  Tasks and timing in the perception of linguistic anomaly , 1996, Journal of psycholinguistic research.

[42]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[43]  William D. Marslen-Wilson,et al.  Lexical Representations in Spoken Language Comprehension , 1988 .

[44]  D. V. von Cramon,et al.  Syntactic comprehension in Parkinson's disease: investigating early automatic and late integrational processes using event-related brain potentials. , 2003, Neuropsychology.

[45]  R. Jackendoff The Status of Thematic Relations in Linguistic Theory , 1987 .

[46]  Phillip J. Holcomb,et al.  Neurocognitive Mechanisms of Human Comprehension , 2008 .

[47]  Leslie G. Ungerleider,et al.  Discrete Cortical Regions Associated with Knowledge of Color and Knowledge of Action , 1995, Science.

[48]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[49]  D. Caplan,et al.  Electrophysiological distinctions in processing conceptual relationships within simple sentences. , 2003, Brain research. Cognitive brain research.

[50]  C. Carter,et al.  Error Detection, Correction, and Prevention in the Brain: A Brief Review of Data and Theories , 2006, Clinical EEG and neuroscience.

[51]  A M Dale,et al.  Optimal experimental design for event‐related fMRI , 1999, Human brain mapping.

[52]  A. Damasio,et al.  Neural Correlates of Naming Actions and of Naming Spatial Relations , 2001, NeuroImage.

[53]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[54]  M. Rugg,et al.  Electrophysiology of Mind: Event-Related Brain Potentials and Cognition , 1995 .

[55]  L. Aravind,et al.  Integration of Word Meaning and World Knowledge in Language Comprehension , 2022 .

[56]  H. Kolk,et al.  Structure and limited capacity in verbal working memory: A study with event-related potentials , 2003, Brain and Language.

[57]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[58]  C. C. Wood,et al.  Event-related potentials, lexical decision and semantic priming. , 1985, Electroencephalography and clinical neurophysiology.

[59]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[60]  M. Rugg The effects of semantic priming and work repetition on event-related potentials. , 1985, Psychophysiology.

[61]  R. Turner,et al.  Event-Related fMRI: Characterizing Differential Responses , 1998, NeuroImage.

[62]  Anders M. Dale,et al.  Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex , 2001, IEEE Transactions on Medical Imaging.

[63]  P. Holcomb,et al.  Frontal and Parietal Components of a Cerebral Network Mediating Voluntary Attention to Novel Events , 2003, Journal of Cognitive Neuroscience.

[64]  Stefanie Shattuck-Hufnagel,et al.  Question/statement judgments: An fMRI study of intonation processing , 2004, Human brain mapping.

[65]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[66]  D Y von Cramon,et al.  Language related brain potentials in patients with cortical and subcortical left hemisphere lesions. , 1999, Brain : a journal of neurology.

[67]  W. Gehring,et al.  Neural Systems for Error Monitoring , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[68]  Sharon L. Thompson-Schill,et al.  Conceptual Representations of Action in the Lateral Temporal Cortex , 2005, Journal of Cognitive Neuroscience.

[69]  G. McCarthy,et al.  Language-related field potentials in the anterior-medial temporal lobe: II. Effects of word type and semantic priming , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[70]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[71]  P. Holcomb,et al.  Event related potentials and language comprehension. , 1995 .

[72]  G. McCarthy,et al.  Language-related field potentials in the anterior-medial temporal lobe: I. Intracranial distribution and neural generators , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[73]  J. Haxby,et al.  Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects , 1999, Nature Neuroscience.

[74]  Lorraine K. Tyler,et al.  Spoken language comprehension : an experimental approach to disordered and normal processing , 1992 .

[75]  Vicka R. Corey,et al.  On the Language Specificity of the Brain Response to Syntactic Anomalies: Is the Syntactic Positive Shift a Member of the P300 Family? , 1996, Journal of Cognitive Neuroscience.

[76]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[77]  Gina R Kuperberg,et al.  Building up linguistic context in schizophrenia: evidence from self-paced reading. , 2006, Neuropsychology.

[78]  M. Ullman A neurocognitive perspective on language: The declarative/procedural model , 2001, Nature Reviews Neuroscience.

[79]  A M Dale,et al.  Estimation and detection of event‐related fMRI signals with temporally correlated noise: A statistically efficient and unbiased approach , 2000, Human brain mapping.

[80]  E. Gibson,et al.  The P600 as an index of syntactic integration difficulty , 2000 .

[81]  P. Holcomb,et al.  In: Understanding Events: How Humans See, Represent, and Act on Events. , 2007 .

[82]  Anders M. Dale,et al.  Improved Localization of Cortical Activity By Combining EEG and MEG with MRI Cortical Surface Reconstruction , 2002 .

[83]  T. Ditman,et al.  An investigation of concurrent ERP and self-paced reading methodologies. , 2007, Psychophysiology.

[84]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[85]  D. Greve,et al.  Task and semantic relationship influence both the polarity and localization of hemodynamic modulation during lexico‐semantic processing , 2008, Human brain mapping.

[86]  Alex Martin,et al.  Semantic memory and the brain: structure and processes , 2001, Current Opinion in Neurobiology.

[87]  Phillip J Holcomb,et al.  Semantic integration in videos of real-world events: an electrophysiological investigation. , 2003, Psychophysiology.

[88]  D. Swinney,et al.  Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing , 2005, Human brain mapping.

[89]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[90]  Phillip J. Holcomb,et al.  Neural correlates of processing syntactic, semantic, and thematic relationships in sentences , 2006 .

[91]  Francesco Vespignani,et al.  Differences in the perception and time course of syntactic and semantic violations , 2003, Brain and Language.

[92]  E. Bullmore,et al.  Common and Distinct Neural Substrates for Pragmatic, Semantic, and Syntactic Processing of Spoken Sentences: An fMRI Study , 2000, Journal of Cognitive Neuroscience.

[93]  D. Perani,et al.  Syntax and the Brain: Disentangling Grammar by Selective Anomalies , 2001, NeuroImage.

[94]  M. Alexander,et al.  Distributed anatomy of transcortical sensory aphasia. , 1989, Archives of neurology.

[95]  H. Kolk,et al.  Late positivities in unusual situations , 2007, Brain and Language.

[96]  P. Hagoort,et al.  A superficial resemblance does not necessarily mean you are part of the family : Counterarguments to Coulson, King and Kutas (1998) in the P600/SPS-P300 debate , 1999 .

[97]  T. Shallice,et al.  Dual-Process Model in Semantic Priming: A Functional Imaging Perspective , 1999, NeuroImage.

[98]  A. Nobre,et al.  The anatomy and time course of semantic priming investigated by fMRI and ERPs , 2003, Neuropsychologia.

[99]  R. Poldrack,et al.  Recovering Meaning Left Prefrontal Cortex Guides Controlled Semantic Retrieval , 2001, Neuron.

[100]  M. Farah,et al.  Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[101]  M. Kutas,et al.  Expect the Unexpected: Event-related Brain Response to Morphosyntactic Violations , 1998 .

[102]  R. Ashton,et al.  Handedness assessment inventory , 1976, Neuropsychologia.

[103]  Irene P. Kan,et al.  Effects of Repetition and Competition on Activity in Left Prefrontal Cortex during Word Generation , 1999, Neuron.

[104]  Jacqueline Gottlieb,et al.  LIP responses to a popout stimulus are reduced if it is overtly ignored , 2006, Nature Neuroscience.

[105]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[106]  Rolf A. Zwaan,et al.  Embodied Language: A Review of the Role of the Motor System in Language Comprehension , 2008, Quarterly journal of experimental psychology.

[107]  G. Rizzolatti,et al.  Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study , 2001, The European journal of neuroscience.

[108]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[109]  C. Fiebach,et al.  The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. , 2003, Cerebral cortex.

[110]  A. Dale,et al.  Distinct Patterns of Neural Modulation during the Processing of Conceptual and Syntactic Anomalies , 2003, Journal of Cognitive Neuroscience.

[111]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[112]  J R Hodges,et al.  Selective impairment of verb processing associated with pathological changes in Brodmann areas 44 and 45 in the motor neurone disease-dementia-aphasia syndrome. , 2001, Brain : a journal of neurology.

[113]  C. Fiebach,et al.  Processing lexical semantic and syntactic information in first and second language: fMRI evidence from German and Russian , 2005, Human brain mapping.

[114]  Gina R. Kuperberg,et al.  Neural mechanisms of language comprehension: Challenges to syntax , 2007, Brain Research.

[115]  J. Trueswell,et al.  Cognitive control and parsing: Reexamining the role of Broca’s area in sentence comprehension , 2005, Cognitive, affective & behavioral neuroscience.

[116]  Shanley E. M. Allen,et al.  The Cambridge Handbook of Child Language: Verb argument structure , 2009 .

[117]  Angela D. Friederici,et al.  Who did what to whom? The neural basis of argument hierarchies during language comprehension , 2005, NeuroImage.

[118]  C. Petten,et al.  Neural localization of semantic context effects in electromagnetic and hemodynamic studies , 2006, Brain and Language.

[119]  A. Syrota,et al.  The Cortical Representation of Speech , 1993, Journal of Cognitive Neuroscience.

[120]  E. Vakil,et al.  Motor and non-motor sequence learning in patients with basal ganglia lesions: the case of serial reaction time (SRT) , 2000, Neuropsychologia.

[121]  David Kemmerer,et al.  Action to Language via the Mirror Neuron System: Action verbs, argument structure constructions, and the mirror neuron system , 2006 .

[122]  Beth Levin,et al.  Argument Realization: Acknowledgments , 2005 .

[123]  M. Arbib Action to language via the mirror neuron system , 2006 .

[124]  L. Osterhout,et al.  The independence of combinatory semantic processing: Evidence from event-related potentials , 2005 .

[125]  Kara D. Federmeier,et al.  Electrophysiology reveals semantic memory use in language comprehension , 2000, Trends in Cognitive Sciences.