Composition of Complex Meaning

Language enables humans to communicate the most complex of thoughts, such that from patterns of sensory stimulation an interlocutor can reconstruct the message originating in a speaker’s brain. While every component of this remarkable process is a difficult problem, the fundamental starting point for this entire sequence of events is our ability to construct a complex meaning for the purpose of expression. Underlying this capacity is a generative system of meaning composition that sits in the interface of language and thought. Decades of research within philosophically based linguistics and in the psychology of concepts have characterized the nature of these complex semantic representations; however, this work has only recently begun to inform the brain science of semantic representation. In this chapter I bring together perspectives from theoretical linguistics, cognitive psychology, and neuropsychology to discuss what is currently known about the brain basis of complex meaning composition. Although differences in cognitive neuroscience techniques and experimental paradigms often result in more differences than similarities, a remarkable convergence arises from hemodynamic methods, magnetoencephalography (MEG), and neuropsychological data implicating the left anterior temporal lobe (LATL) as a core region for the construction of complex meaning. The focus of this chapter is on MEG data, on the basis of which the timing of LATL activity in comprehension has been characterized as relatively early, peaking at 200–250 ms after the onset of a composing word. Progress in our understanding of the computational contribution of this activity is only beginning to gain momentum, but I summarize an initial body of results suggesting that, crucially, the LATL supports automatic, task-insensitive composition in both comprehension and production and that its contribution is clearly semantic, as opposed to syntactic in nature.

[1]  Carlo Semenza,et al.  The Syntactic and Semantic Processing of Mass and Count Nouns: An ERP Study , 2011, PloS one.

[2]  J. Hodges,et al.  Charting the progression in semantic dementia: implications for the organisation of semantic memory. , 1995 .

[3]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[4]  Peter Hagoort,et al.  The neurobiology of language beyond single words. , 2014, Annual review of neuroscience.

[5]  Martin J. Pickering,et al.  Coercion in sentence processing: evidence from eye-movements and self-paced reading , 2002 .

[6]  R. Shayna Rosenbaum,et al.  Episodic and Semantic Memory , 2017 .

[7]  M. Tarr,et al.  Levels of categorization in visual recognition studied using functional magnetic resonance imaging , 1997, Current Biology.

[8]  Gregory L. Murphy,et al.  Noun phrase interpretation and conceptual combination , 1990 .

[9]  P. Fletcher,et al.  DO SEMANTIC CATEGORIES ACTIVATE DISTINCT CORTICAL REGIONS? EVIDENCE FOR A DISTRIBUTED NEURAL SEMANTIC SYSTEM , 2003, Cognitive neuropsychology.

[10]  Soheyl Noachtar,et al.  Naming tools and animals: asymmetries observed during direct electrical cortical stimulation , 2002, Neuropsychologia.

[11]  L. Pylkkänen,et al.  Processing psych verbs: Behavioural and MEG measures of two different types of semantic complexity , 2010 .

[12]  J. Hodges,et al.  Semantic dementia. Progressive fluent aphasia with temporal lobe atrophy. , 1992 .

[13]  M N Rossor,et al.  Patterns of temporal lobe atrophy in semantic dementia and Alzheimer's disease , 2001, Annals of neurology.

[14]  Liina Pylkkänen,et al.  Simple Composition: A Magnetoencephalography Investigation into the Comprehension of Minimal Linguistic Phrases , 2011, The Journal of Neuroscience.

[15]  Gennaro Chierchia,et al.  Meaning and Grammar: An Introduction to Semantics , 1990 .

[16]  Benjamin Cohen,et al.  Models of Concepts , 1984, Cogn. Sci..

[17]  Nancy Kanwisher,et al.  Language-Selective and Domain-General Regions Lie Side by Side within Broca’s Area , 2012, Current Biology.

[18]  R. Ilmoniemi,et al.  Language-specific phoneme representations revealed by electric and magnetic brain responses , 1997, Nature.

[19]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[20]  A. Nobre,et al.  The Response of Left Temporal Cortex to Sentences , 2002, Journal of Cognitive Neuroscience.

[21]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[22]  Emmanuel A Stamatakis,et al.  The Anatomy of Object Processing: The Role of Anteromedial Temporal Cortex , 2005, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[23]  P. Hagoort,et al.  Integration of Word Meaning and World Knowledge in Language Comprehension , 2004, Science.

[24]  M. Garrett,et al.  Syntactically Based Sentence Processing Classes: Evidence from Event-Related Brain Potentials , 1991, Journal of Cognitive Neuroscience.

[25]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[26]  M. Tanenhaus,et al.  Circumscribing Referential Domains during Real-Time Language Comprehension , 2002 .

[27]  H. Kamp,et al.  Prototype theory and compositionality , 1995, Cognition.

[28]  A. Friederici,et al.  Auditory Language Comprehension: An Event-Related fMRI Study on the Processing of Syntactic and Lexical Information , 2000, Brain and Language.

[29]  Daniel N. Osherson,et al.  Evidence for conceptual combination in the left anterior temporal lobe , 2011, NeuroImage.

[30]  Matthew H. Davis,et al.  Is there an anatomical basis for category-specificity? Semantic memory studies in PET and fMRI , 2002, Neuropsychologia.

[31]  D. Medin,et al.  Context and structure in conceptual combination , 1988, Cognitive Psychology.

[32]  Richard Montague,et al.  ENGLISH AS A FORMAL LANGUAGE , 1975 .

[33]  James L. McClelland,et al.  Structure and deterioration of semantic memory: a neuropsychological and computational investigation. , 2004, Psychological review.

[34]  Liina Pylkkänen,et al.  Flexible Composition: Meg Evidence for the Deployment of Basic Combinatorial Linguistic Mechanisms in Response to Task Demands Flexible Composition (meg) , 2022 .

[35]  Richard Montague,et al.  The Proper Treatment of Quantification in Ordinary English , 1973 .

[36]  Nancy Kanwisher,et al.  Broad domain generality in focal regions of frontal and parietal cortex , 2013, Proceedings of the National Academy of Sciences.

[37]  Elizabeth Jefferies,et al.  Both the Middle Temporal Gyrus and the Ventral Anterior Temporal Area Are Crucial for Multimodal Semantic Processing: Distortion-corrected fMRI Evidence for a Double Gradient of Information Convergence in the Temporal Lobes , 2012, Journal of Cognitive Neuroscience.

[38]  A. Friederici The cortical language circuit: from auditory perception to sentence comprehension , 2012, Trends in Cognitive Sciences.

[39]  Roy W Jones,et al.  Presemantic Cognition in Semantic Dementia: Six Deficits in Search of an Explanation , 2006 .

[40]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[41]  Gregory Hickok,et al.  Selective attention to semantic and syntactic features modulates sentence processing networks in anterior temporal cortex. , 2009, Cerebral cortex.

[42]  Liina Pylkkänen,et al.  The time-course and spatial distribution of brain activity associated with sentence processing , 2012, NeuroImage.

[43]  Yosef Grodzinsky,et al.  Neuroimaging of syntax and syntactic processing , 2006, Current Opinion in Neurobiology.

[44]  Edward E. Smith,et al.  Conceptual Combination with Prototype Concepts , 1984, Cogn. Sci..

[45]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[46]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[47]  L. Pylkkänen,et al.  Basic linguistic composition recruits the left anterior temporal lobe and left angular gyrus during both listening and reading. , 2013, Cerebral cortex.

[48]  Liina Pylkkänen,et al.  Mismatching Meanings in Brain and Behavior , 2008, Lang. Linguistics Compass.

[49]  Jiang Xu,et al.  Language in context: emergent features of word, sentence, and narrative comprehension , 2005, NeuroImage.

[50]  L. Pylkkänen,et al.  Semantics vs. world knowledge in prefrontal cortex , 2009 .

[51]  A. Newman,et al.  How the mass counts: An electrophysiological approach to the processing of lexical features , 2001, Neuroreport.

[52]  Peter Hagoort,et al.  Retrieval and unification of syntactic structure in sentence comprehension: an FMRI study using word-category ambiguity. , 2009, Cerebral cortex.

[53]  Paolo Vitali,et al.  Generating animal and tool names: An fMRI study of effective connectivity , 2005, Brain and Language.

[54]  Andrea E. Martin,et al.  The Anterior Midline Field: coercion or decision making? , 2009 .

[55]  Richard J. Binney,et al.  The ventral and inferolateral aspects of the anterior temporal lobe are crucial in semantic memory: evidence from a novel direct comparison of distortion-corrected fMRI, rTMS, and semantic dementia. , 2010, Cerebral cortex.

[56]  A. A. Wijers,et al.  Localizing components of a complex task: sentence processing and working memory , 1998, Neuroreport.

[57]  Karl G. D. Bailey,et al.  Good-Enough Representations in Language Comprehension , 2002 .

[58]  L. Pylkkänen,et al.  Restrictive vs. non-restrictive composition: a magnetoencephalography study , 2014, Language, cognition and neuroscience.

[59]  A. Syrota,et al.  The Cortical Representation of Speech , 1993, Journal of Cognitive Neuroscience.

[60]  Godehard Link The Logical Analysis of Plurals and Mass Terms: A Lattice‐theoretical Approach , 2008 .

[61]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[62]  Gregory L. Murphy,et al.  Comprehending Complex Concepts , 1988, Cogn. Sci..

[63]  Mark T. Keane,et al.  Efficient creativity: Constraint-guided conceptual combination. , 2000 .

[64]  Liina Pylkkänen,et al.  Grounding the cognitive neuroscience of semantics in linguistic theory , 2011 .

[65]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[66]  D. Swinney,et al.  Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing , 2005, Human brain mapping.

[67]  E. Tulving,et al.  Episodic and semantic memory , 1972 .

[68]  Liina Pylkkänen,et al.  The role of the left anterior temporal lobe in semantic composition vs. semantic memory , 2014, Neuropsychologia.

[69]  Liina Pylkkänen,et al.  An MEG Study of Silent Meaning , 2007, Journal of Cognitive Neuroscience.

[70]  E. Wisniewski Construal and Similarity in Conceptual Combination , 1996 .

[71]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[72]  Christopher Kennedy Vagueness and grammar: the semantics of relative and absolute gradable adjectives , 2007 .

[73]  Richard S. J. Frackowiak,et al.  A voxel‐based morphometry study of semantic dementia: Relationship between temporal lobe atrophy and semantic memory , 2000, Annals of neurology.

[74]  Paul Wright,et al.  Objects and Categories: Feature Statistics and Object Processing in the Ventral Stream , 2013, Journal of Cognitive Neuroscience.

[75]  C. Price,et al.  Disrupted temporal lobe connections in semantic dementia. , 1999 .

[76]  D. John Done and Tim M. Gale,et al.  Attribute Verification in Dementia of Alzheimer Type: Evidence for the Preservation of Distributed Concept Knowledge , 1997 .

[77]  Fernanda Ferreira,et al.  The 'Good Enough' Approach to Language Comprehension , 2007, Lang. Linguistics Compass.

[78]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[79]  Rutvik H. Desai,et al.  The neurobiology of semantic memory , 2011, Trends in Cognitive Sciences.

[80]  N. Alpert,et al.  Localization of Syntactic Comprehension by Positron Emission Tomography , 1996, Brain and Language.

[81]  Martin J. Pickering,et al.  A time course analysis of enriched composition , 2006, Psychonomic bulletin & review.

[82]  Brian McElree,et al.  Complement Coercion Is Not Modulated by Competition: Evidence from Eye Movements , 2022 .

[83]  T. Rogers,et al.  Anterior temporal cortex and semantic memory: Reconciling findings from neuropsychology and functional imaging , 2006, Cognitive, affective & behavioral neuroscience.

[84]  Robin K. Morris,et al.  Eye movements and on-line language comprehension processes , 1989 .

[85]  Lisa Matthewson,et al.  Quantification and the Nature of Crosslinguistic Variation , 2001 .

[86]  G. Dehaene-Lambertz Electrophysiological correlates of categorical phoneme perception in adults , 1997, Neuroreport.

[87]  Liina Pylkkänen,et al.  MEG evidence for conceptual combination but not numeral quantification in the left anterior temporal lobe during language production , 2014, Front. Psychol..

[88]  Liina Pylkkänen,et al.  Processing events: Behavioral and neuromagnetic correlates of Aspectual Coercion , 2008, Brain and Language.

[89]  J. Rodd,et al.  Anteromedial temporal cortex supports fine-grained differentiation among objects. , 2005, Cerebral cortex.

[90]  G. Chierchia,et al.  Reference to Kinds across Language , 1998 .

[91]  Bernard Mazoyer,et al.  Impact of modality and linguistic complexity during reading and listening tasks , 2007, NeuroImage.

[92]  Alex Clarke,et al.  The Evolution of Meaning: Spatio-temporal Dynamics of Visual Object Recognition , 2011, Journal of Cognitive Neuroscience.

[93]  Uta Noppeney,et al.  Retrieval of abstract semantics , 2004, NeuroImage.

[94]  Liina Pylkkänen,et al.  The LATL as locus of composition: MEG evidence from English and Arabic , 2015, Brain and Language.

[95]  J. Desmond,et al.  Functional Specialization for Semantic and Phonological Processing in the Left Inferior Prefrontal Cortex , 1999, NeuroImage.

[96]  D. Saumier,et al.  Semantic and syntactic aspects of the mass/count distinction: A case study of semantic dementia , 2005, Brain and Cognition.

[97]  Nora Fieder,et al.  From “some butter” to “a butter”: An investigation of mass and count representation and processing , 2014, Cognitive neuropsychology.

[98]  Colin Humphries,et al.  Syntactic and Semantic Modulation of Neural Activity during Auditory Sentence Comprehension , 2006, Journal of Cognitive Neuroscience.

[99]  E. Warrington Quarterly Journal of Experimental Psychology the Selective Impairment of Semantic Memory the Selective Impairment of Semantic Memory , 2022 .

[100]  D. Aaronson,et al.  Performance theories for sentence coding: Some quantitative evidence. , 1976 .

[101]  Liina Pylkkänen,et al.  Building phrases in language production: An MEG study of simple composition , 2014, Cognition.

[102]  J. Rodd,et al.  Processing Objects at Different Levels of Specificity , 2004, Journal of Cognitive Neuroscience.

[103]  Maria Bittner,et al.  Cross-linguistic semantics , 1994 .

[104]  Kara D. Federmeier,et al.  Electrophysiology reveals semantic memory use in language comprehension , 2000, Trends in Cognitive Sciences.

[105]  Edward E. Smith,et al.  On the adequacy of prototype theory as a theory of concepts , 1981, Cognition.