Modelling of syntactical processing in the cortex

Probably the hardest test for a theory of brain function is the explanation of language processing in the human brain, in particular the interplay of syntax and semantics. Clearly such an explanation can only be very speculative, because there are essentially no animal models and it is hard to study detailed neural processing in humans. The approach presented in this paper uses well established basic neural mechanisms in a plausible global network architecture that is formulated essentially in terms of cortical areas and their intracortical and corticocortical interconnections. The neural implementation of this system shows that the comparatively intricate logical task of understanding semantico-syntactical structures can be mastered by a neural network architecture. The system presented also shows additional context awareness, in particular the model is able to correct ambiguous input to a certain degree, e.g. the input "bot show/lift green wall" with an artificial ambiguity between "show" and "lift" is correctly interpreted as "bot show green wall" since a wall is not liftable. Furthermore, the system is able to learn new object words during runtime.

[1]  Michael A. Arbib,et al.  Synthetic brain imaging: grasping, mirror neurons and imitation , 2000, Neural Networks.

[2]  A. Knoblauch,et al.  Associative Language Processing in Cortical Areas , 2005 .

[3]  Stevan Harnad.  The Symbol Grounding Problem , 1999, ArXiv.

[4]  S. Nelson,et al.  An emergent model of orientation selectivity in cat visual cortical simple cells , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Prof. Dr. Valentino Braitenberg,et al.  Anatomy of the Cortex , 1991, Studies of Brain Function.

[6]  Günther Palm.  Rules for synaptic changes and their relevance for the storage of information in the brain , 1982 .

[7]  C. von der Malsburg,et al.  Am I Thinking Assemblies , 1986 .

[8]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[9]  G. Palm,et al.  Associating words to visually recognized objects ∗ , 2004 .

[10]  Stefan Wermter.  Hybrid Connectionist Natural Language Processing , 1994 .

[11]  W. M. Keck,et al.  Machine Psychology : Autonomous Behavior , Perceptual Categorization and Conditioning in a Brain-based Device , 2002 .

[12]  E. Vaadia,et al.  Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. , 1993, Journal of neurophysiology.

[13]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[14]  Don R. Hush,et al.  Bounds on the complexity of recurrent neural network implementations of finite state machines , 1993, Neural Networks.

[15]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[16]  Braitenberg,et al.  Brain theory : biological basis and computational principles , 1996 .

[17]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[18]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[19]  Friedemann Pulvermüller,et al.  The Neuroscience of Language: On Brain Circuits of Words and Serial Order , 2003 .

[20]  Sukhdev Khebbal,et al.  Intelligent Hybrid Systems , 1994 .

[21]  A. Miyake,et al.  Models of Working Memory: Mechanisms of Active Maintenance and Executive Control , 1999 .

[22]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[23]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[24]  C. Lee Giles,et al.  Extraction, Insertion and Refinement of Symbolic Rules in Dynamically Driven Recurrent Neural Networks , 1993 .

[25]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[26]  Günther Palm,et al.  Biomimetic Neural Learning for Intelligent Robots - Intelligent Systems, Cognitive Robotics, and Neuroscience , 2005, Biomimetic Neural Learning for Intelligent Robots.

[27]  Alessandro Sperduti,et al.  On the implementation of frontier-to-root tree automata in recursive neural networks , 1999, IEEE Trans. Neural Networks.

[28]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[29]  Nils J. Nilsson,et al.  Artificial Intelligence: A New Synthesis , 1997 .

[30]  Lokendra Shastri,et al.  Rules and Variables in Neural Nets , 1991, Neural Computation.

[31]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[32]  Günther Palm,et al.  Local rules for synaptic modification in neural networks , 1993 .

[33]  J. Tsien,et al.  Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes , 2006, Trends in Neurosciences.

[34]  P. Frasconi,et al.  Representation of Finite State Automata in Recurrent Radial Basis Function Networks , 1996, Machine Learning.

[35]  Gèunther Palm,et al.  Neural Assemblies: An Alternative Approach to Artificial Intelligence , 1982 .

[36]  Ron Sun,et al.  Computational Architectures Integrating Neural And Symbolic Processes , 1994 .

[37]  Hava T. Siegelmann,et al.  The complexity of language recognition by neural networks , 1992, Neurocomputing.

[38]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[39]  S. Grossberg,et al.  Texture segregation, surface representation and figure–ground separation , 1998, Vision Research.

[40]  Marco Gori,et al.  Adaptive Processing of Sequences and Data Structures , 1998, Lecture Notes in Computer Science.

[41]  Alberto Sanfeliu,et al.  An Algebraic Framework to Represent Finite State Machines in Single-Layer Recurrent Neural Networks , 1995, Neural Computation.

[42]  Günther Palm,et al.  Associative Networks and Cell Assemblies , 1986 .

[43]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[44]  Scott T. Grafton,et al.  Synthetic PET imaging for grasping: from primate Neurophysiology to human behavior , 2003 .

[45]  Jonathan D. Cohen,et al.  A Biologically Based Computational Model of Working Memory , 1999 .

[46]  Rodney A. Brooks,et al.  Building brains for bodies , 1995, Auton. Robots.

[47]  Friedemann Pulvermüller.  Sequence detectors as a basis of grammar in the brain , 2003 .

[48]  Aude Billard,et al.  DRAMA, a Connectionist Architecture for Control and Learning in Autonomous Robots , 1999, Adapt. Behav..

[49]  Professor Dr. Valentino Braitenberg,et al.  On the Texture of Brains , 1977, Heidelberg Science Library.

[50]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[51]  Karl J. Friston,et al.  Neural modeling and functional brain imaging: an overview , 2000, Neural Networks.

[52]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[53]  R. Eckhorn,et al.  Coherent oscillations: A mechanism of feature linking in the visual cortex? , 1988, Biological Cybernetics.

[54]  Jeffrey D. Ullman,et al.  Formal languages and their relation to automata , 1969, Addison-Wesley series in computer science and information processing.

[55]  Gerald M Edelman,et al.  A cerebellar model for predictive motor control tested in a brain-based device. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[57]  Risto Miikkulainen,et al.  Parsing Embedded Clauses with Distributed Neural Networks , 1994, AAAI.

[58]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[59]  Eduardo D. Sontag,et al.  Analog Neural Nets with Gaussian or Other Common Noise Distributions Cannot Recognize Arbitrary Regular Languages , 1999, Neural Computation.

[60]  M Abeles,et al.  Spatio-temporal firing patterns in the frontal cortex of behaving monkeys , 1996, Journal of Physiology-Paris.

[61]  José R. Álvarez,et al.  Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired Approach: First International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2005, Las Palmas, Canary Islands, Spain, June 15-18, 2005, Proceedings, Part II , 2005, IWINAC.

[62]  Günther Palm,et al.  On the Information Storage Capacity of Local Learning Rules , 1992, Neural Computation.

[63]  Luc Steels,et al.  The artificial life route to artificial intelligence : building embodied , 1995 .

[64]  Günther Palm,et al.  Combining Visual Attention, Object Recognition and Associative Information Processing in a NeuroBotic System , 2005, Biomimetic Neural Learning for Intelligent Robots.

[65]  G Palm,et al.  Computing with neural networks. , 1987, Science.

[66]  Andreas Knoblauch,et al.  Synchronization and pattern separation in spiking associative memories and visual cortical areas , 2004 .

[67]  S. Levin.  Lectu re Notes in Biomathematics , 1983 .

[68]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[69]  Günther Palm,et al.  An Associative Cortical Model of Language Understanding and Action Planning , 2005, IWINAC.

[70]  Günther Palm,et al.  Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states , 1992 .

[71]  Günther Palm,et al.  Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales , 2002, Biological Cybernetics.

[72]  Günther Palm,et al.  Detecting Sequences and Understanding Language with Neural Associative Memories and Cell Assemblies , 2005, Biomimetic Neural Learning for Intelligent Robots.

[73]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.