Spatiotemporal Novelty Detection Using Resonance Networks

We present a single-layer recurrent neural network that implements novelty detection for spatiotemporal patterns. The architecture is based on the structure of region CA3 in the hippocampus, which is believed to implement this function. Through analysis and numerical simulation we generate theorems that constrain the operation of this network. We show that once a pattern has been encoded, it will never be miscategorized as novel. We also show that the upper bound on the network capacity is equal to the number of connections. We discuss the tradeoff between generalization and total performance in the architecture.

[1]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[2]  S Dehaene,et al.  Neural networks that learn temporal sequences by selection. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[3]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[4]  D. Spalding.  The Principles of Psychology , 1873, Nature.

[5]  Stephen Marsland,et al.  On-Line Novelty Detection through self-organisation with application to inspection robotics , 2001 .

[6]  J. Lisman,et al.  Hippocampus as comparator: Role of the two input and two output systems of the hippocampus in selection and registration of information , 2001, Hippocampus.

[7]  R. Passingham.  The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[8]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[9]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[10]  Martin Schneider,et al.  Activity-Dependent Development of Axonal and Dendritic Delays, or, Why Synaptic Transmission Should Be Unreliable , 2002, Neural Computation.

[11]  E. Oja,et al.  Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuron-like elements , 1976, Biological Cybernetics.

[12]  L. Swanson.  The anatomical organization of septo-hippocampal projections. , 1977, Ciba Foundation symposium.

[13]  S Dehaene,et al.  Spin glass model of learning by selection. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[15]  Szabolcs Káli,et al.  A familiarity-based learning procedure for the establishment of place fields in area CA3 of the rat hippocampus , 2001, Neurocomputing.

[16]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[17]  H. V. Restorff.  Über die Wirkung von Bereichsbildungen im Spurenfeld , 1933 .

[18]  G. Rainer,et al.  Cognitive neuroscience: Neural mechanisms for detecting and remembering novel events , 2003, Nature Reviews Neuroscience.

[19]  Christophe Giraud-Carrier,et al.  High Capacity Neural Networks for Familiarity Discrimination , 1999 .

[20]  Alberto Muñoz,et al.  Self-organizing maps for outlier detection , 1998, Neurocomputing.

[21]  Stephen Grossberg,et al.  ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures , 1990, Neural Networks.