Neural correlate of the construction of sentence meaning

Significance How do circuits of neurons in your brain extract and hold the meaning of a sentence? To start to address this unanswered question, we measured neural activity from the surface of the human brain in patients being mapped out before neurosurgery, as they read sentences. In many electrodes, neural activity increased steadily over the course of the sentence, but the same was not found when participants read lists of words or pronounceable nonwords, or grammatical nonword strings (“Jabberwocky”). This build-up of neural activity appears to reflect neither word meaning nor syntax alone, but the representation of complex meanings. The neural processes that underlie your ability to read and understand this sentence are unknown. Sentence comprehension occurs very rapidly, and can only be understood at a mechanistic level by discovering the precise sequence of underlying computational and neural events. However, we have no continuous and online neural measure of sentence processing with high spatial and temporal resolution. Here we report just such a measure: intracranial recordings from the surface of the human brain show that neural activity, indexed by γ-power, increases monotonically over the course of a sentence as people read it. This steady increase in activity is absent when people read and remember nonword-lists, despite the higher cognitive demand entailed, ruling out accounts in terms of generic attention, working memory, and cognitive load. Response increases are lower for sentence structure without meaning (“Jabberwocky” sentences) and word meaning without sentence structure (word-lists), showing that this effect is not explained by responses to syntax or word meaning alone. Instead, the full effect is found only for sentences, implicating compositional processes of sentence understanding, a striking and unique feature of human language not shared with animal communication systems. This work opens up new avenues for investigating the sequence of neural events that underlie the construction of linguistic meaning.

[1]  D. Osherson,et al.  The boundaries of language and thought in deductive inference , 2009, Proceedings of the National Academy of Sciences.

[2]  Rolf A. Zwaan,et al.  The Construction of Situation Models in Narrative Comprehension: An Event-Indexing Model , 1995 .

[3]  P. Holcomb,et al.  Event-related brain potentials elicited by syntactic anomaly , 1992 .

[4]  Daniel N. Osherson,et al.  Evidence for conceptual combination in the left anterior temporal lobe , 2011, NeuroImage.

[5]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[6]  E. Chang,et al.  Human cortical sensorimotor network underlying feedback control of vocal pitch , 2013, Proceedings of the National Academy of Sciences.

[7]  R. Knight,et al.  Redefining the role of Broca’s area in speech , 2015, Proceedings of the National Academy of Sciences.

[8]  Gerwin Schalk,et al.  A Practical Guide to Brain–Computer Interfacing with BCI2000: General-Purpose Software for Brain-Computer Interface Research, Data Acquisition, Stimulus Presentation, and Brain Monitoring , 2010 .

[9]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[10]  Morton Ann Gernsbacher,et al.  Language Comprehension As Structure Building , 1990 .

[11]  P. Hagoort The fractionation of spoken language understanding by measuring electrical and magnetic brain signals , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[12]  D. A. Robertson,et al.  Functional Neuroanatomy of the Cognitive Process of Mapping During Discourse Comprehension , 2000, Psychological science.

[13]  N. Kanwisher,et al.  New method for fMRI investigations of language: defining ROIs functionally in individual subjects. , 2010, Journal of neurophysiology.

[14]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[15]  Rolf A. Zwaan,et al.  Discourse comprehension. , 1997, Annual review of psychology.

[16]  C. Van Petten,et al.  Prediction during language comprehension: benefits, costs, and ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[17]  Nancy Kanwisher,et al.  Broad domain generality in focal regions of frontal and parietal cortex , 2013, Proceedings of the National Academy of Sciences.

[18]  Alexa Bautista,et al.  Neural responses to grammatically and lexically degraded speech , 2016, Language, cognition and neuroscience.

[19]  Bradley Greger,et al.  Decoding spoken words using local field potentials recorded from the cortical surface , 2010, Journal of neural engineering.

[20]  Michael J. Kahana,et al.  Direct brain recordings fuel advances in cognitive electrophysiology , 2010, Trends in Cognitive Sciences.

[21]  Daniel N. Osherson,et al.  Thought Beyond Language , 2012, Psychological science.

[22]  T. Trabasso,et al.  Constructing inferences during narrative text comprehension. , 1994, Psychological review.

[23]  A. Dale,et al.  Distinct Patterns of Neural Modulation during the Processing of Conceptual and Syntactic Anomalies , 2003, Journal of Cognitive Neuroscience.

[24]  N. Geschwind The Organization of Language and the Brain: Language disorders after brain damage help in elucidating the neural basis of verbal behavior , 1970 .

[25]  A. Nobre,et al.  The Response of Left Temporal Cortex to Sentences , 2002, Journal of Cognitive Neuroscience.

[26]  Matthew K. Leonard,et al.  Dynamic Encoding of Speech Sequence Probability in Human Temporal Cortex , 2015, The Journal of Neuroscience.

[27]  Edith Kaan,et al.  Event-Related Potentials and Language Processing: A Brief Overview , 2007, Lang. Linguistics Compass.

[28]  Eishi Asano,et al.  Gamma activity modulated by picture and auditory naming tasks: Intracranial recording in patients with focal epilepsy , 2013, Clinical Neurophysiology.

[29]  Edward F Chang,et al.  Control of Spoken Vowel Acoustics and the Influence of Phonetic Context in Human Speech Sensorimotor Cortex , 2014, The Journal of Neuroscience.

[30]  E. Chang,et al.  Categorical Speech Representation in Human Superior Temporal Gyrus , 2010, Nature Neuroscience.

[31]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[32]  Wilhelm Freiherr von Humboldt,et al.  On Language: On the Diversity of Human Language Construction and Its Influence on the Mental Development of the Human Species , 2001 .

[33]  Peter Hagoort,et al.  Retrieval and unification of syntactic structure in sentence comprehension: an FMRI study using word-category ambiguity. , 2009, Cerebral cortex.

[34]  Charles A. Perfetti,et al.  There are Generalized Abilities and One of Them is Reading , 2018, Knowing, Learning, and Instruction.

[35]  Philippe Kahane,et al.  Cortical dynamics of word recognition , 2008, Human brain mapping.

[36]  Anders M. Dale,et al.  N400-like Magnetoencephalography Responses Modulated by Semantic Context, Word Frequency, and Lexical Class in Sentences , 2002, NeuroImage.

[37]  G. Schalk,et al.  Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans , 2011, Journal of neural engineering.

[38]  Jeremy R. Manning,et al.  Broadband Shifts in Local Field Potential Power Spectra Are Correlated with Single-Neuron Spiking in Humans , 2009, The Journal of Neuroscience.

[39]  Eric Halgren,et al.  Sequential Processing of Lexical, Grammatical, and Phonological Information Within Broca’s Area , 2009, Science.

[40]  E. F. Chang,et al.  Sub-centimeter language organization in the human temporal lobe , 2011, Brain and Language.

[41]  Stephen M. Rao,et al.  Human Brain Language Areas Identified by Functional Magnetic Resonance Imaging , 1997, The Journal of Neuroscience.

[42]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[43]  R. Saxe,et al.  Language processing in the occipital cortex of congenitally blind adults , 2011, Proceedings of the National Academy of Sciences.

[44]  Nancy Kanwisher,et al.  Language-Selective and Domain-General Regions Lie Side by Side within Broca’s Area , 2012, Current Biology.

[45]  N. Logothetis,et al.  Frequency-Band Coupling in Surface EEG Reflects Spiking Activity in Monkey Visual Cortex , 2009, Neuron.

[46]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[47]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[48]  WILLIAM MARSLEN-WILSON,et al.  Processing structure of sentence perception , 1975, Nature.

[49]  Bijan Pesaran,et al.  Sensory-motor transformations for speech occur bilaterally , 2014, Nature.

[50]  N. Kanwisher,et al.  A functional dissociation between language and multiple-demand systems revealed in patterns of BOLD signal fluctuations. , 2014, Journal of neurophysiology.

[51]  J. Duncan The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour , 2010, Trends in Cognitive Sciences.

[52]  Nancy Kanwisher,et al.  Functional specificity for high-level linguistic processing in the human brain , 2011, Proceedings of the National Academy of Sciences.

[53]  J. Maunsell,et al.  Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex , 2011, PLoS biology.

[54]  Yosef Grodzinsky,et al.  Neuroimaging of syntax and syntactic processing , 2006, Current Opinion in Neurobiology.

[55]  Rocky Ross,et al.  Mental models , 2004, SIGA.

[56]  N. Kanwisher,et al.  Lexical and syntactic representations in the brain: An fMRI investigation with multi-voxel pattern analyses , 2011, Neuropsychologia.

[57]  Robert T. Knight,et al.  Spatiotemporal imaging of cortical activation during verb generation and picture naming , 2010, NeuroImage.

[58]  Kristofer E. Bouchard,et al.  Functional Organization of Human Sensorimotor Cortex for Speech Articulation , 2013, Nature.

[59]  Jeffrey G. Ojemann,et al.  Power-Law Scaling in the Brain Surface Electric Potential , 2009, PLoS Comput. Biol..

[60]  D. Shankweiler,et al.  Unification of sentence processing via ear and eye: An fMRI study , 2011, Cortex.

[61]  Matthew K. Leonard,et al.  The influence of lexical statistics on temporal lobe cortical dynamics during spoken word listening , 2015, Brain and Language.

[62]  A. Anwander,et al.  The brain differentiates human and non-human grammars: Functional localization and structural connectivity , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Nitin Tandon,et al.  Category specific spatial dissociations of parallel processes underlying visual naming. , 2014, Cerebral cortex.

[64]  Evelina Fedorenko,et al.  Syntactic processing is distributed across the language system , 2016, NeuroImage.

[65]  I. Fried,et al.  Human intracranial recordings and cognitive neuroscience. , 2012, Annual review of psychology.

[66]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[67]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[68]  Nathaniel J. Smith,et al.  The effect of word predictability on reading time is logarithmic , 2013, Cognition.

[69]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[70]  Michael H Kohrman,et al.  ECoG gamma activity during a language task: differentiating expressive and receptive speech areas. , 2008, Brain : a journal of neurology.

[71]  N. Geschwind The organization of language and the brain. , 1970, Science.

[72]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[73]  P. Chauvel,et al.  Ventral and dorsal pathways of speech perception: An intracerebral ERP study , 2013, Brain and Language.

[74]  D L Miglioretti,et al.  Temporal dynamics of verbal object comprehension. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[75]  Arnaud Delorme,et al.  High-Frequency γ-Band Activity in the Basal Temporal Cortex during Picture-Naming and Lexical-Decision Tasks , 2005, The Journal of Neuroscience.

[76]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[77]  C. J. Price,et al.  An fMRI Study of Syntactic Adaptation , 2004, Journal of Cognitive Neuroscience.

[78]  P. Hagoort On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[79]  L. Pylkkänen,et al.  Basic linguistic composition recruits the left anterior temporal lobe and left angular gyrus during both listening and reading. , 2013, Cerebral cortex.

[80]  F. Dick,et al.  Voxel-based lesion–symptom mapping , 2003, Nature Neuroscience.

[81]  M. Ross Quillian,et al.  The teachable language comprehender: a simulation program and theory of language , 1969, CACM.

[82]  R. Malach,et al.  Syntactic structure building in the anterior temporal lobe during natural story listening , 2012, Brain and Language.

[83]  Eishi Asano,et al.  Cortical gamma oscillations modulated by word association tasks: Intracranial recording , 2010, Epilepsy & Behavior.

[84]  Colin M. Brown,et al.  The syntactic positive shift (sps) as an erp measure of syntactic processing , 1993 .

[85]  Jed A. Meltzer,et al.  Localization of electrophysiological responses to semantic and syntactic anomalies in language comprehension with MEG , 2015, NeuroImage.

[86]  A. Graesser Prose Comprehension Beyond the Word , 1981 .

[87]  J P Rauschecker,et al.  Hemispheric specialization for English and ASL: left invariance‐right variability , 1998, Neuroreport.