Prolegomena to a Neurocomputational Architecture for Human Grammatical Encoding and Decoding

This study develops a neurocomputational architecture for grammatical processing in language production and language comprehension (grammatical encoding and decoding, respectively). It seeks to answer two questions. First, how is online syntactic structure formation of the complexity required by natural-language grammars possible in a fixed, preexisting neural network without the need for online creation of new connections or associations? Second, is it realistic to assume that the seemingly disparate instantiations of syntactic structure formation in grammatical encoding and grammatical decoding can run on the same neural infrastructure? This issue is prompted by accumulating experimental evidence for the hypothesis that the mechanisms for grammatical decoding overlap with those for grammatical encoding to a considerable extent, thus inviting the hypothesis of a single “grammatical coder.” The paper answers both questions by providing the blueprint for a syntactic structure formation mechanism that is entirely based on prewired circuitry (except for referential processing, which relies on the rapid learning capacity of the hippocampal complex), and can subserve decoding as well as encoding tasks. The model builds on the “Unification Space” model of syntactic parsing developed by Vosse and Kempen (Cognition 75:105–143, 2000; Cognitive Neurodynamics 3:331–346, 2009a). The design includes a neurocomputational mechanism for the treatment of an important class of grammatical movement phenomena.

[1]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[2]  Suzanne Stevenson,et al.  A Competition-Based Explanation of Syntactic Attachment Preferences and Garden Path Phenomena , 1993, ACL.

[3]  Anton Nijholt,et al.  Computational Linguistics in the Netherlands 2001 , 2002 .

[4]  K. Holyoak,et al.  A symbolic-connectionist theory of relational inference and generalization. , 2003, Psychological review.

[5]  Gerard Kempen,et al.  Could grammatical encoding and grammatical decoding be subserved by the same processing module? , 2000, Behavioral and Brain Sciences.

[6]  S. Köhler,et al.  Why is the meaning of a sentence better remembered than its form? An fMRI study on the role of novelty‐encoding processes , 2008, Hippocampus.

[7]  Pentti Kanerva,et al.  Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors , 2009, Cognitive Computation.

[8]  P. Hagoort On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[9]  Antje S. Meyer,et al.  Automaticity and control in Language Processing , 2007 .

[10]  James L. McClelland,et al.  Generalization Through the Recurrent Interaction of Episodic Memories , 2012, Psychological review.

[11]  Giorgio A. Ascoli,et al.  Adaptive Recall in Hippocampus , 2011, BICA.

[12]  J. Trueswell,et al.  Cognitive control and parsing: Reexamining the role of Broca’s area in sentence comprehension , 2005, Cognitive, affective & behavioral neuroscience.

[13]  Paul Thagard,et al.  The AHA! Experience: Creativity Through Emergent Binding in Neural Networks , 2011, Cogn. Sci..

[14]  G. Dell,et al.  Is comprehension necessary for error detection? A conflict-based account of monitoring in speech production , 2011, Cognitive Psychology.

[15]  Kathryn Bock,et al.  Making syntax of sense: number agreement in sentence production. , 2005, Psychological review.

[16]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[17]  Roberto Pirrone,et al.  Biologically Inspired Cognitive Architectures 2012 - Proceedings of the Third Annual Meeting of the BICA Society, Palermo, Sicily, Italy, October 31 - November 3, 2012 , 2012, BICA.

[18]  D. Kumaran,et al.  Which computational mechanisms operate in the hippocampus during novelty detection? , 2007, Hippocampus.

[19]  Ross W. Gayler Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience , 2004, ArXiv.

[20]  Dylan M. Jones,et al.  Short Article: Syntax and Serial Recall: How Language Supports Short-Term Memory for Order , 2009, Quarterly journal of experimental psychology.

[21]  Michael D. Howard,et al.  Complementary Learning Systems , 2014, Cogn. Sci..

[22]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[23]  Peter Hagoort,et al.  How the brain solves the binding problem for language: a neurocomputational model of syntactic processing , 2003, NeuroImage.

[24]  John C. Trueswell,et al.  Co-localization of Stroop and Syntactic Ambiguity Resolution in Broca's Area: Implications for the Neural Basis of Sentence Processing , 2009, Journal of Cognitive Neuroscience.

[25]  Karin Harbusch,et al.  6. Dutch and German verb constructions in Performance Grammar , 2003 .

[26]  Giosuè Baggio,et al.  The balance between memory and unification in semantics: A dynamic account of the N400 , 2011 .

[27]  Gerard Kempen,et al.  Incremental Sentence Generation: Implications for the Structure of a Syntactic Processor , 1982, COLING.

[28]  Donald G. MacKay,et al.  Compensating for Language Deficits in Amnesia I: H.M.’s Spared Retrieval Categories , 2013, Brain sciences.

[29]  Gregory Hickok,et al.  Left posterior auditory-related cortices participate both in speech perception and speech production: Neural overlap revealed by fMRI , 2006, Brain and Language.

[30]  Gerard Kempen,et al.  In Defense of Competition During Syntactic Ambiguity Resolution , 2008, Journal of psycholinguistic research.

[31]  Ronald M. Kaplan,et al.  Augmented Transition Networks as Psychological Models of Sentence Comprehension , 1971, IJCAI.

[32]  B. MacWhinney,et al.  Vocabulary Acquisition and Verbal Short-Term Memory: Computational and Neural Bases , 1997, Brain and Language.

[33]  Angela D. Friederici,et al.  Linking ordering in Broca's area to storage in left temporo-parietal regions: The case of sentence processing , 2012, NeuroImage.

[34]  David R. Dowty,et al.  Natural Language Parsing: Psychological, Computational, and Theoretical Perspectives , 1985 .

[35]  Peter Hagoort,et al.  The memory, unification, and control (MUC) model of language , 2007 .

[36]  F. van der Velde,et al.  Neural blackboard architectures of combinatorial structures in cognition , 2006, Behavioral and Brain Sciences.

[37]  A. Glenberg,et al.  Mental models contribute to foregrounding during text comprehension , 1987 .

[38]  Julie A. Hengst,et al.  The Use of Definite References Signals Declarative Memory , 2011, Psychological science.

[39]  Nelson Cowan,et al.  Working Memory Capacity , 2005 .

[40]  Gerard Kempen,et al.  Verb Constructions in German and Dutch , 2003 .

[41]  Sarah Brown-Schmidt,et al.  The hippocampus and the flexible use and processing of language , 2012, Front. Hum. Neurosci..

[42]  Jinyong Lee,et al.  Template Construction Grammar: From Visual Scene Description to Language Comprehension and Agrammatism , 2013, Neuroinformatics.

[43]  Geoffrey E. Hinton Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[44]  Sam Featherston,et al.  Bridge verbs and V2 verbs – the same thing in spades? , 2004 .

[45]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[46]  J. Elman Distributed representations, simple recurrent networks, and grammatical structure , 1991, Machine Learning.

[47]  Tessa C. Warren,et al.  READING-TIME EVIDENCE FOR INTERMEDIATE LINGUISTIC STRUCTURE IN LONG-DISTANCE DEPENDENCIES , 2004 .

[48]  Hartmut Fitz,et al.  Getting real about Semantic Illusions: Rethinking the functional role of the P600 in language comprehension , 2012, Brain Research.

[49]  Sarah M. E. Gierhan,et al.  Shared Language , 2011, Psychological science.

[50]  Gerard Kempen,et al.  Parsing Verb-Final Clauses in German: Garden-path and ERP Effects Modeled by a Parallel Dynamic Parser , 2008 .

[51]  Rachel A. Diana,et al.  Imaging recollection and familiarity in the medial temporal lobe: a three-component model , 2007, Trends in Cognitive Sciences.

[52]  Daniel J. Acheson,et al.  Verbal working memory and language production: Common approaches to the serial ordering of verbal information. , 2009, Psychological bulletin.

[53]  Michael A. Arbib,et al.  The metaphorical brain 2 - neural networks and beyond (2. ed.) , 1972 .

[54]  Peter Hagoort,et al.  The neurobiology of syntax: beyond string sets , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[55]  Gerard Kempen,et al.  An Incremental Procedural Grammar for Sentence Formulation , 1987, Cogn. Sci..

[56]  Gerard Kempen,et al.  The Unification Space implemented as a localist neural net: predictions and error-tolerance in a constraint-based parser , 2009, Cognitive Neurodynamics.

[57]  A. Baddeley Working memory: theories, models, and controversies. , 2012, Annual review of psychology.

[58]  Peter Hagoort,et al.  Shared Syntax in Language Production and Language Comprehension—An fMRI Study , 2011, Cerebral cortex.

[59]  Gerard Kempen,et al.  Incremental syntactic tree formation in human sentence processing: A cognitive architecture based on activation decay and simulated annealing , 1989 .

[60]  J. Sachs Recognition memory for syntactic and semantic aspects of connected discourse , 1967 .

[61]  Jerome Feldman,et al.  The neural binding problem(s) , 2013, Cognitive Neurodynamics.

[62]  M. F. Garrett,et al.  The Analysis of Sentence Production1 , 1975 .

[63]  Markus Werning,et al.  Compositionality and Biologically Plausible Models , 2009 .

[64]  Karin Harbusch,et al.  A Quantitative Model of Word Order and Movement in English, Dutch and German Complement Constructions , 2002, COLING.

[65]  Simone Sprenger,et al.  Grammatical workspace sharing during language production and language comprehension: Evidence from grammatical multitasking , 2012 .

[66]  Markus Werning,et al.  The Oxford Handbook of Compositionality , 2012 .

[67]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[68]  Jared M. Novick,et al.  Broca's Area and Language Processing: Evidence for the Cognitive Control Connection , 2010, Lang. Linguistics Compass.

[69]  A. Friederici The brain basis of language processing: from structure to function. , 2011, Physiological reviews.

[70]  Karin Harbusch,et al.  Performance Grammar: a Declarative Definition , 2001, CLIN.

[71]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[72]  A. Friederici The cortical language circuit: from auditory perception to sentence comprehension , 2012, Trends in Cognitive Sciences.

[73]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[74]  Donald G. MacKay,et al.  Compensating for Language Deficits in Amnesia II: H.M.’s Spared versus Impaired Encoding Categories , 2013, Brain sciences.

[75]  F. Pulvermüller Brain embodiment of syntax and grammar: Discrete combinatorial mechanisms spelt out in neuronal circuits , 2010, Brain and Language.

[76]  M. Schlesewsky,et al.  Reconciling time, space and function: A new dorsal–ventral stream model of sentence comprehension , 2013, Brain and Language.

[77]  Daniel J. Acheson,et al.  A Common Neural Substrate for Language Production and Verbal Working Memory , 2011, Journal of Cognitive Neuroscience.

[78]  Shravan Vasishth,et al.  Towards dynamical system models of language-related brain potentials , 2008, Cognitive Neurodynamics.

[79]  P. Johnson-Laird How We Reason , 2006 .

[80]  G. Kempen,et al.  Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar , 2000, Cognition.

[81]  Gerard Kempen The unification space: A hybrid model of human syntactic processing [Abstract] , 1994 .

[82]  Pieter R. Roelfsema,et al.  The Brain's Router: A Cortical Network Model of Serial Processing in the Primate Brain , 2010, PLoS Comput. Biol..

[83]  Lila Davachi,et al.  Evidence for area CA1 as a match/mismatch detector: A high‐resolution fMRI study of the human hippocampus , 2012, Hippocampus.

[84]  James Henderson,et al.  Connectionist syntactic parsing using temporal variable binding , 1994 .

[85]  K.J.M.J. De Smedt Incremental sentence generation: a computer model of grammatical encoding , 1990 .

[86]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[87]  Peter Hagoort,et al.  Retrieval and unification of syntactic structure in sentence comprehension: an FMRI study using word-category ambiguity. , 2009, Cerebral cortex.

[88]  Bradley R. Buchsbaum,et al.  The Search for the Phonological Store: From Loop to Convolution , 2008, Journal of Cognitive Neuroscience.

[89]  Peter beim Graben,et al.  Unifying syntactic theory and sentence processing difficulty through a connectionist minimalist parser , 2009, Cognitive Neurodynamics.

[90]  Adrian Staub,et al.  Eye movements and processing difficulty in object relative clauses , 2010, Cognition.

[91]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[92]  Peter Ford Dominey,et al.  Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing , 2013, PloS one.

[93]  J. Trueswell,et al.  Parametric effects of syntactic–semantic conflict in Broca’s area during sentence processing , 2012, Brain and Language.

[94]  Noam Chomsky,et al.  Birdsong, Speech, and Language: Exploring the Evolution of Mind and Brain , 2013 .

[95]  Gary F. Marcus,et al.  How Does the Mind Work? Insights from Biology , 2009, Top. Cogn. Sci..

[96]  William A. Woods,et al.  Computational Linguistics Transition Network Grammars for Natural Language Analysis , 2022 .

[97]  Ina Bornkessel-Schlesewsky,et al.  Towards a Computational Model of Actor-Based Language Comprehension , 2013, Neuroinformatics.

[98]  J. Nicol,et al.  Separating hierarchical relations and word order in language production: is proximity concord syntactic or linear? , 1998, Cognition.

[99]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[100]  Aravind K. Joshi,et al.  Natural language parsing: Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? , 1985 .

[101]  R. Saxe,et al.  Look at this: the neural correlates of initiating and responding to bids for joint attention , 2012, Front. Hum. Neurosci..

[102]  N. Burgess,et al.  The hippocampus and memory: insights from spatial processing , 2008, Nature Reviews Neuroscience.

[103]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[104]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[105]  Bernadette M. Jansma,et al.  Neural Basis of Linearization in Speech Production , 2011, Journal of Cognitive Neuroscience.

[106]  Ina Bornkessel-Schlesewsky,et al.  The Role of Prominence Information in the Real-Time Comprehension of Transitive Constructions: A Cross-Linguistic Approach , 2009, Lang. Linguistics Compass.

[107]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..