Creating hierarchical categories using cell assemblies

Highly recurrent neural networks can learn reverberating circuits called Cell Assemblies (CAs). These networks can be used to categorize input, and this paper explores the ability of CAs to learn hierarchical categories. A simulator, based on spiking fatiguing leaky integrators, is presented with instances of base categories. Learning is done using a compensatory Hebbian learning rule. The model takes advantage of overlapping CAs where neurons may participate in more than one CA. Using the unsupervised compensatory learning rule, the networks learn a hierarchy of categories that correctly categorize 97% of the basic level presentations of the input in our test. It categorizes 100% of the super-categories correctly. A larger hierarchy is learned that correctly categorizes 100% of base categories, and 89% of super-categories. It is also shown how novel subcategories gain default information from their super-category. These simulations show that networks containing CAs can be used to learn hierarchical categories. The network then can successfully categorize novel inputs.

[1]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[2]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. , 1982, Psychological review.

[3]  W. Gerstner,et al.  Time structure of the activity in neural network models. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[4]  Matthew L. Shapiro,et al.  Simulating Hebb cell assemblies: the necessity for partitioned dendritic trees and a post-not-pre LTD rule , 1993 .

[5]  M R Quillian,et al.  Word concepts: a theory and simulation of some basic semantic capabilities. , 1967, Behavioral science.

[6]  H. Barlow,et al.  Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology? , 1972, Perception.

[7]  Claire Cardie.  Using Cognitive Biases to Guide Feature Set Selection , 1992 .

[8]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[9]  Eric Chown,et al.  Making predictions in an uncertain world: Environmental structure and cognitive maps , 1999, Adapt. Behav..

[10]  E. Vaadia,et al.  Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. , 1993, Journal of neurophysiology.

[11]  Christian R. Huyck.  Overlapping cell assemblies from correlators , 2004, Neurocomputing.

[12]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[13]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[14]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[15]  Eric Chown,et al.  Tracing Recurrent Activity in Cognitive Elements (TRACE): a Model of Temporal Dynamics in a Cell Assembly , 1991 .

[16]  Daniel J. Amit,et al.  Spike-Driven Synaptic Dynamics Generating Working Memory States , 2003, Neural Computation.

[17]  Ron Sun,et al.  Robust Reasoning: Integrating Rule-Based and Similarity-Based Reasoning , 1995, Artif. Intell..

[18]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[19]  J. Bower,et al.  The Book of GENESIS , 1998, Springer New York.

[20]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[21]  Stephen José Hanson,et al.  Conceptual Clustering, Categorization, and Polymorphy , 1989, Machine Learning.

[22]  Andreas Knoblauch,et al.  An associative model of cortical language and action processing , 2005 .

[23]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[24]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[25]  Heinrich Klar,et al.  TOWARDS EFFICIENT HARDWARE FOR SPIKE-PROCESSING NEURAL NETWORKS , 1995 .

[26]  Jacques P Sougné,et al.  Binding and multiple instantiation in a distributed network of spiking nodes , 2001, Connect. Sci..

[27]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[28]  Christian R. Huyck,et al.  Counting with neurons: rule application with nets of fatiguing leaking integrate and fire neurons. , 2006 .

[29]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[30]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[31]  J. Kruschke,et al.  Rules and exemplars in category learning. , 1998, Journal of experimental psychology. General.

[32]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[33]  Mats Rooth,et al.  Structural Ambiguity and Lexical Relations , 1991, ACL.

[34]  W. James.  Psychology: Briefer Course , 2020 .

[35]  V. Braitenberg,et al.  Some Arguments for a Theory of Cell Assemblies in the Cerebral Cortex , 1989 .

[36]  W. Freeman.  Mesoscopic neurodynamics: From neuron to brain , 2000, Journal of Physiology-Paris.

[37]  C. Wilson,et al.  Mechanisms Underlying Spontaneous Oscillation and Rhythmic Firing in Rat Subthalamic Neurons , 1999, The Journal of Neuroscience.

[38]  Anthony M. Zador,et al.  Novel Integrate-and-re-like Model of Repetitive Firing in Cortical Neurons , 1998 .

[39]  R. Shepard,et al.  Learning and memorization of classifications. , 1961 .

[40]  W A Wickelgren,et al.  Webs, cell assemblies, and chunking in neural nets: introduction. , 1999, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[41]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[42]  J. Holland,et al.  An analysis of Hebb's cell assembly as a mechanism for perceptual generalization , 2003 .

[43]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[44]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[45]  Lev B. Ioffe,et al.  The Augmented Models of Associative Memory Asymmetric Interaction and Hierarchy of Patterns - Int. J. Mod. Phys. B1, 51 (1987) , 1987 .

[46]  John C. Eccles,et al.  Chapter 1 Chemical transmission and Dale's principle , 1986 .

[47]  J. Fodor.  The Mind Doesn't Work That Way : The Scope and Limits of Computational Psychology , 2000 .

[48]  Nicolas Brunel,et al.  Hebbian Learning of Context in Recurrent Neural Networks , 1996, Neural Computation.

[49]  G. Athithan.  ASSOCIATIVE MEMORY OF LOW ACTIVITY PATTERNS WITHOUT THE PROBLEM OF SPURIOUS ATTRACTORS , 1999 .

[50]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[51]  Eytan Ruppin,et al.  Memory Maintenance via Neuronal Regulation , 1998, Neural Computation.

[52]  Eric L. Schwartz,et al.  Computing with the Leaky Integrate-and-Fire Neuron: Logarithmic Computation and Multiplication , 1997, Neural Computation.

[53]  David E. Kieras,et al.  Predictive engineering models based on the EPIC architecture for a multimodal high-performance human-computer interaction task , 1997, TCHI.

[54]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[55]  J. Sougné.  Binding and Multiple Instantiation in a Distributed Network of Spiking Neurons , 2000 .

[56]  John H. Holland,et al.  Tests on a cell assembly theory of the action of the brain, using a large digital computer , 1956, IRE Trans. Inf. Theory.

[57]  Alessandro Treves,et al.  An associative network with spatially organized connectivity , 2004 .

[58]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[59]  Marius Usher,et al.  Neural Network Modeling of Memory Deterioration in Alzheimer's Disease , 1993, Neural Computation.

[60]  R. Reilly,et al.  A PROPOSED MODEL OF REPETITION BLINDNESS , 2005 .

[61]  Christian R. Huyck,et al.  Information Retrieval and Categorisation using a Cell Assembly Network , 2005, Neural Computing & Applications.

[62]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[63]  Jordan B. Pollack,et al.  Infinite RAAM: A Principled Connectionist Basis for Grammatical Competence , 2000 .

[64]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[65]  Christian R. Huyck,et al.  Cell Assemblies as an Intermediate Level Model of Cognition , 2001, Emergent Neural Computational Architectures Based on Neuroscience.

[66]  Christian R. Huyck,et al.  IMPROVING CELL ASSEMBLY CATEGORIES BY FATIGUE , 2005 .

[67]  A. Newell.  Unified Theories of Cognition , 1990 .

[68]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[69]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[70]  Hans Liljenström,et al.  A model of cortical associative memory based on Hebbian cell assemblies , 1994 .

[71]  William H. Calvin.  Cortical columns, modules, and Hebbian cell assemblies , 1998 .

[72]  Horn,et al.  Neural networks with dynamical thresholds. , 1989, Physical review. A, General physics.

[73]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[74]  Stephen Jose Hanson,et al.  On the Emergence of Rules in Neural Networks , 2002, Neural Computation.

[75]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[76]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[77]  Yoshio Sakurai,et al.  The search for cell assemblies in the working brain , 1998, Behavioural Brain Research.

[78]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[79]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.