Complex temporal sequence learning based on short-term memory

An approach to storing of temporal sequences that deals with complex temporal sequences directly is presented. Short-term memory (STM) is modeled by units comprised of recurrent excitatory connections between two neurons. A dual-neuron model is proposed. By applying the Hebbian learning rule at each synapse and a normalization rule among all synaptic weights of a neuron, it is shown that a quantity called the input potential increases monotonically with sequence presentation, and that the neuron can only be fired when its input signals are arranged in a specific sequence. These sequence-detecting neurons form the basis for a model of complex sequence recognition that can tolerate distortions of the learned sequences. A recurrent network of two layers is provided for reproducing complex sequences. >

[1]  S. Grossberg.  Some Networks That Can Learn, Remember, and Reproduce any Number of Complicated Space-Time Patterns, I , 1969 .

[2]  William L. Kilmer,et al.  A Wave Model of Temporal Sequence Learning , 1975, Int. J. Man Mach. Stud..

[3]  H. Simon,et al.  Models of Thought , 1979 .

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  S. Amari,et al.  Competition and Cooperation in Neural Nets , 1982 .

[6]  M. Arbib,et al.  The role of the tectal column in facilitation of amphibian prey- catching behavior: a neural model , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[7]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[8]  D Kleinfeld,et al.  Sequential state generation by model neural networks. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[9]  S Dehaene,et al.  Neural networks that learn temporal sequences by selection. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Joachim M. Buhmann,et al.  Noise-driven temporal association in neural networks , 1987 .

[11]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Personnaz,et al.  Storage and retrieval of complex sequences in neural networks. , 1988, Physical review. A, General physics.

[13]  Terrence J. Sejnowski,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cognitive Sciences.

[14]  Gutfreund,et al.  Processing of temporal sequences in neural networks. , 1988, Physical review letters.

[15]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[16]  Joachim M. Buhmann,et al.  Pattern Segmentation in Associative Memory , 1990, Neural Computation.