A psycholinguistic model of natural language parsing implemented in simulated neurons

A natural language parser implemented entirely in simulated neurons is described. It produces a semantic representation based on frames. It parses solely using simulated fatiguing Leaky Integrate and Fire neurons, that are a relatively accurate biological model that is simulated efficiently. The model works on discrete cycles that simulate 10 ms of biological time, so the parser has a simple mapping to psychological parsing time. Comparisons to human parsing studies show that the parser closely approximates this data. The parser makes use of Cell Assemblies and the semantics of lexical items is represented by overlapping hierarchical Cell Assemblies so that semantically related items share neurons. This semantic encoding is used to resolve prepositional phrase attachment ambiguities encountered during parsing. Consequently, the parser provides a neurally-based cognitive model of parsing.

[1]  Günther Palm,et al.  Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals , 2000 .

[2]  E. Rolls.  Memory, Attention, and Decision-Making: A unifying computational neuroscience approach , 2007 .

[3]  Brian Roark,et al.  Efficient probabilistic top-down and left-corner parsing , 1999, ACL.

[4]  Robin Cooper,et al.  Incremental Interpretation: Applications, Theory, And Relationship To Dynamic Semantics , 1994, COLING.

[5]  Christian R. Huyck,et al.  Counting with neurons: rule application with nets of fatiguing leaking integrate and fire neurons. , 2006 .

[6]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[7]  Roger P. G. van Gompel,et al.  Reanalysis in Sentence Processing: Evidence against Current Constraint-Based and Two-Stage Models , 2001 .

[8]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[9]  G. Striedter.  Précis of Principles of Brain Evolution , 2006, Behavioral and Brain Sciences.

[10]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[11]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[12]  C. von der Malsburg,et al.  Am I Thinking Assemblies , 1986 .

[13]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[14]  Yulei Fan,et al.  Implementation of finite state automata using fLIF neurons , 2008, 2008 7th IEEE International Conference on Cybernetic Intelligent Systems.

[15]  Andreas Knoblauch,et al.  An associative model of cortical language and action processing , 2005 .

[16]  Emmon W. Bach,et al.  Universals in Linguistic Theory , 1970 .

[17]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[18]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[19]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[20]  Adwait Ratnaparkhi,et al.  A Maximum Entropy Model for Prepositional Phrase Attachment , 1994, HLT.

[21]  Christian R. Huyck,et al.  Prepositional phrase attachment ambiguity resolution using semantic hierarchies , 2009 .

[22]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[23]  Modelling Cell Assemblies , 1999 .

[24]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[25]  Preslav Nakov,et al.  Using the Web as an Implicit Training Set: Application to Structural Ambiguity Resolution , 2005, HLT.

[26]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[27]  IEEE Systems , Man , and Cybernetics Society ’ s Continuing Legacy in Human – Machine Systems , .

[28]  Gerard Kempen,et al.  Incremental syntactic tree formation in human sentence processing: A cognitive architecture based on activation decay and simulated annealing , 1989 .

[29]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[30]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[31]  Michael K. Tanenhaus,et al.  Dynamical models of sentence processing , 1999, Cogn. Sci..

[32]  Christian R. Huyck,et al.  Emergence of Rules in Cell Assemblies of fLIF Neurons , 2008, ECAI.

[33]  R. Jackendoff.  Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[34]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[35]  Eric L. Schwartz,et al.  Computing with the Leaky Integrate-and-Fire Neuron: Logarithmic Computation and Multiplication , 1997, Neural Computation.

[36]  John R. Anderson.  How Can the Human Mind Occur in the Physical Universe , 2007 .

[37]  Charles J. Fillmore,et al.  THE CASE FOR CASE. , 1967 .

[38]  Eric Chown,et al.  Tracing Recurrent Activity in Cognitive Elements (TRACE): a Model of Temporal Dynamics in a Cell Assembly , 1991 .

[39]  C. Wilson,et al.  Mechanisms Underlying Spontaneous Oscillation and Rhythmic Firing in Rat Subthalamic Neurons , 1999, The Journal of Neuroscience.

[40]  Christian R. Huyck,et al.  Creating hierarchical categories using cell assemblies , 2007, Connect. Sci..

[41]  S. Nelson,et al.  Multiple forms of short-term plasticity at excitatory synapses in rat medial prefrontal cortex. , 2000, Journal of neurophysiology.

[42]  C. Pollard,et al.  Center for the Study of Language and Information , 2022 .

[43]  F. van der Velde,et al.  Neural blackboard architectures of combinatorial structures in cognition , 2006, Behavioral and Brain Sciences.

[44]  Friedemann Pulvermüller,et al.  Syntactic Circuits: How Does the Brain Create Serial Order in Sentences? , 2000, Brain and Language.

[45]  CurteanuNeculai.  Review of "Lecture on contemporary syntactic theories , 1987 .

[46]  Christian Huyck,et al.  Parsing with fLIF Neurons , 2007 .

[47]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[48]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[49]  André Longtin,et al.  Interspike Interval Correlations, Memory, Adaptation, and Refractoriness in a Leaky Integrate-and-Fire Model with Threshold Fatigue , 2003, Neural Computation.

[50]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[51]  C. Huyck,et al.  CABot1: A videogame agent implemented in fLIF neurons , 2008, 2008 7th IEEE International Conference on Cybernetic Intelligent Systems.

[52]  Stuart M. Shieber,et al.  An Introduction to Unification-Based Approaches to Grammar , 1986, CSLI Lecture Notes.

[53]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[54]  Ray Jackendoff,et al.  Précis of Foundations of language: brain, meaning, grammar, evolution. , 2003, The Behavioral and brain sciences.

[55]  James Henderson,et al.  Connectionist syntactic parsing using temporal variable binding , 1994 .

[56]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[57]  D. Buonomano,et al.  Distinct Functional Types of Associative Long-Term Potentiation in Neocortical and Hippocampal Pyramidal Neurons , 1999, The Journal of Neuroscience.

[58]  Risto Miikkulainen,et al.  Subsymbolic natural language processing - an integrated model of scripts, lexicon, and memory , 1993, Neural network modeling and connectionism.

[59]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[60]  Ivan A. Sag,et al.  Book Reviews: Head-driven Phrase Structure Grammar and German in Head-driven Phrase-structure Grammar , 1996, CL.

[61]  P.J. Passmore,et al.  Models of cell assembly decay , 2008, 2008 7th IEEE International Conference on Cybernetic Intelligent Systems.

[62]  Z. Harris,et al.  Foundations of language , 1941 .

[63]  E. Vaadia,et al.  Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. , 1993, Journal of neurophysiology.