Neurophysiological dynamics of phrase-structure building during sentence processing

Significance According to most linguists, the syntactic structure of sentences involves a tree-like hierarchy of nested phrases, as in the sentence [happy linguists] [draw [a diagram]]. Here, we searched for the neural implementation of this hypothetical construct. Epileptic patients volunteered to perform a language task while implanted with intracranial electrodes for clinical purposes. While patients read sentences one word at a time, neural activation in left-hemisphere language areas increased with each successive word but decreased suddenly whenever words could be merged into a phrase. This may be the neural footprint of “merge,” a fundamental tree-building operation that has been hypothesized to allow for the recursive properties of human language. Although sentences unfold sequentially, one word at a time, most linguistic theories propose that their underlying syntactic structure involves a tree of nested phrases rather than a linear sequence of words. Whether and how the brain builds such structures, however, remains largely unknown. Here, we used human intracranial recordings and visual word-by-word presentation of sentences and word lists to investigate how left-hemispheric brain activity varies during the formation of phrase structures. In a broad set of language-related areas, comprising multiple superior temporal and inferior frontal sites, high-gamma power increased with each successive word in a sentence but decreased suddenly whenever words could be merged into a phrase. Regression analyses showed that each additional word or multiword phrase contributed a similar amount of additional brain activity, providing evidence for a merge operation that applies equally to linguistic objects of arbitrary complexity. More superficial models of language, based solely on sequential transition probability over lexical and syntactic categories, only captured activity in the posterior middle temporal gyrus. Formal model comparison indicated that the model of multiword phrase construction provided a better fit than probability-based models at most sites in superior temporal and inferior frontal cortices. Activity in those regions was consistent with a neural implementation of a bottom-up or left-corner parser of the incoming language stream. Our results provide initial intracranial evidence for the neurophysiological reality of the merge operation postulated by linguists and suggest that the brain compresses syntactically well-formed sequences of words into a hierarchy of nested phrases.

[1]  Peter Hagoort,et al.  Syntactic Unification Operations Are Reflected in Oscillatory Dynamics during On-line Sentence Comprehension , 2010, Journal of Cognitive Neuroscience.

[2]  Florent Meyniel,et al.  The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees , 2015, Neuron.

[3]  Morten H. Christiansen,et al.  Learning Simple Statistics for Language Comprehension and Production: The CAPPUCCINO Model , 2011, CogSci.

[4]  Jayaram Chandrashekar,et al.  Sequential Processing of Lexical, Grammatical, and Phonological Information Within Broca's Area , 2009 .

[5]  L. Tyler,et al.  Grammatical categories in the brain: the role of morphological structure. , 2007, Cerebral cortex.

[6]  Uta Noppeney,et al.  When sentences live up to your expectations , 2016, NeuroImage.

[7]  D. Perani,et al.  Syntax and the Brain: Disentangling Grammar by Selective Anomalies , 2001, NeuroImage.

[8]  Ray Jackendoff,et al.  X Syntax: A Study of Phrase Structure , 1980 .

[9]  Jeffrey G. Ojemann,et al.  Power-Law Scaling in the Brain Surface Electric Potential , 2009, PLoS Comput. Biol..

[10]  Frank Nuessel,et al.  X̄ syntax: a study of phrase structure. Linguistic Inquiry Monograph Two: Ray Jackendoff, MIT Press, Cambridge, Mass., 1977. xii, 249 pp. $11.50 (subscribers to Linguistic Inquiry pay $9.50). , 1979 .

[11]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[12]  Edward P. Stabler,et al.  An Introduction To Syntactic Analysis And Theory , 2016 .

[13]  C. Pollard,et al.  Center for the Study of Language and Information , 2022 .

[14]  Jesper Andersson,et al.  A multi-modal parcellation of human cerebral cortex , 2016, Nature.

[15]  Philip N. Johnson-Laird,et al.  The computer and the mind - An introduction to cognitive science [Book Review] , 1989, Proceedings of the IEEE.

[16]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[17]  Mariano Sigman,et al.  The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers , 2017, PLoS Comput. Biol..

[18]  Antal van den Bosch,et al.  Prediction During Natural Language Comprehension. , 2016, Cerebral cortex.

[19]  John Hale,et al.  Automaton Theories of Human Sentence Comprehension , 2014 .

[20]  Ellen F. Lau,et al.  The Yin and the Yang of Prediction: An fMRI Study of Semantic Predictive Processing , 2016, PloS one.

[21]  D. Zaykin,et al.  Optimally weighted Z‐test is a powerful method for combining probabilities in meta‐analysis , 2011, Journal of evolutionary biology.

[22]  S. Thompson-Schill,et al.  Reworking the language network , 2014, Trends in Cognitive Sciences.

[23]  Erik D. Reichle,et al.  Investigating the causes of wrap-up effects: Evidence from eye movements and E–Z Reader , 2009, Cognition.

[24]  Géraldine Legendre,et al.  The Harmonic Mind: From Neural Computation to Optimality-Theoretic GrammarVolume I: Cognitive Architecture (Bradford Books) , 2006 .

[25]  F. Mathy,et al.  What’s magic about magic numbers? Chunking and data compression in short-term memory , 2012, Cognition.

[26]  Noam Chomsky,et al.  Problems of projection , 2013 .

[27]  Noam Chomsky,et al.  Evolution, brain, and the nature of language , 2013, Trends in Cognitive Sciences.

[28]  Morten H. Christiansen,et al.  How hierarchical is language use? , 2012, Proceedings of the Royal Society B: Biological Sciences.

[29]  R. Malach,et al.  Syntactic structure building in the anterior temporal lobe during natural story listening , 2012, Brain and Language.

[30]  P. Hagoort On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[31]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[32]  T G Bever,et al.  Syntactic Structure Modifies Attention during Speech Perception and Recognition , 1969, The Quarterly journal of experimental psychology.

[33]  Yosef Grodzinsky,et al.  Neural correlates of syntactic movement: converging evidence from two fMRI experiments , 2004, NeuroImage.

[34]  E. Warrington,et al.  Cognitive Neuropsychology: A Clinical Introduction , 1990 .

[35]  Angela D. Friederici,et al.  The language skeleton after dissecting meaning: A functional segregation within Broca's Area , 2015, NeuroImage.

[36]  Sonja Grün,et al.  Hybrid Scheme for Modeling Local Field Potentials from Point-Neuron Networks , 2015, BMC Neuroscience.

[37]  Jack J. Lin,et al.  Direct brain recordings reveal hippocampal rhythm underpinnings of language processing , 2016, Proceedings of the National Academy of Sciences.

[38]  C. Büchel,et al.  Broca's area and the language instinct , 2003, Nature Neuroscience.

[39]  Nancy Kanwisher,et al.  Neural correlate of the construction of sentence meaning , 2016, Proceedings of the National Academy of Sciences.

[40]  Bernard Mazoyer,et al.  Impact of modality and linguistic complexity during reading and listening tasks , 2007, NeuroImage.

[41]  J. Maunsell,et al.  Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex , 2011, PLoS biology.

[42]  Helen L Jamison,et al.  Morphology and the internal structure of words. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Liliane Haegeman,et al.  Thinking Syntactically: A Guide to Argumentation and Analysis , 2005 .

[44]  Steven M Frankland,et al.  An architecture for encoding sentence meaning in left mid-superior temporal cortex , 2015, Proceedings of the National Academy of Sciences.

[45]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[46]  Geoffrey E. Hinton Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[47]  Herbert N. Clark The psychology of language: An introduction to psycholinguistics and generative grammar. , 1975 .

[48]  W Tecumseh Fitch,et al.  Toward a computational framework for cognitive biology: unifying approaches from cognitive neuroscience and comparative cognition. , 2014, Physics of life reviews.

[49]  Erez Lieberman Aiden,et al.  Quantitative Analysis of Culture Using Millions of Digitized Books , 2010, Science.

[50]  Jean-Marie Annoni,et al.  The Neural Cost of the Auditory Perception of Language Switches: An Event-Related Functional Magnetic Resonance Imaging Study in Bilinguals , 2007, The Journal of Neuroscience.

[51]  W. Luh,et al.  Abstract linguistic structure correlates with temporal activity during naturalistic comprehension , 2016, Brain and Language.

[52]  Naoki Masuda,et al.  A pairwise maximum entropy model accurately describes resting-state human brain networks , 2013, Nature Communications.

[53]  John M. Henderson,et al.  Language structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading , 2016, NeuroImage.

[54]  Trevor Bekolay,et al.  Neural representations of compositional structures: representing and manipulating vector spaces with spiking neurons , 2011, Connect. Sci..

[55]  Stanislas Dehaene,et al.  Subliminal Convergence of Kanji and Kana Words: Further Evidence for Functional Parcellation of the Posterior Temporal Cortex in Visual Word Perception , 2005, Journal of Cognitive Neuroscience.

[56]  Thomas G. Bever,et al.  The underlying structures of sentences are the primary units of immediate speech processing , 1969 .

[57]  Stanislas Dehaene,et al.  A Temporal Bottleneck in the Language Comprehension Network , 2012, The Journal of Neuroscience.

[58]  N. Wetherick The Computer and the Mind: An Introduction to Cognitive Science. By Philip N. Johnson‐Laird. Pp. 444. (Harvard University Press, 1988.) £23.50. (Also available from Fontana (1988), £6.95.) , 1988 .

[59]  S. Frank,et al.  The ERP response to the amount of information conveyed by words in sentences , 2015, Brain and Language.

[60]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[61]  John Hale,et al.  Information-theoretical Complexity Metrics , 2016, Lang. Linguistics Compass.

[62]  Lukas Furst,et al.  X Syntax A Study Of Phrase Structure , 2016 .

[63]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[64]  C. Honey,et al.  Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story , 2011, The Journal of Neuroscience.

[65]  L. Pylkkänen,et al.  Basic linguistic composition recruits the left anterior temporal lobe and left angular gyrus during both listening and reading. , 2013, Cerebral cortex.