An artificial neural network system for temporal-spatial sequence processing

Abstract An artificial network is described that can learn, recognize, and generate higher-order temporal-spatial sequences. It consists of three parts: (1) comparator units, (2) a parallel array of artificial neural networks that are derived from the visual-vestibular networks of the snail Hermissenda , as well as hippocampal neuroanatomy, and (3) delayed feedback lines from the output of the system to the neural network layer. Its advantages include short training time, fast and accurate retrievals, toleration of spatial noise and temporal gaps in test sequences, and ability to store a large number of temporal sequences consisting of non-orthogonal spatial patterns.

[1]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[2]  K. Oka,et al.  Long-term transformation of an inhibitory into an excitatory GABAergic synaptic response. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Lipo Wang,et al.  Artificial neural networks - oscillations, chaos, and sequence processing , 1993 .

[4]  L. Wang,et al.  Oscillations and chaos in neural networks: an exactly solvable model. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[5]  D L Alkon,et al.  Memory storage and neural systems. , 1989, Scientific American.

[6]  S. Grossberg.  Some Networks That Can Learn, Remember, and Reproduce any Number of Complicated Space-Time Patterns, I , 1969 .

[7]  Françoise Fogelman-Soulié,et al.  Disordered Systems and Biological Organization , 1986, NATO ASI Series.

[8]  Joachim M. Buhmann,et al.  Noise-driven temporal association in neural networks , 1987 .

[9]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[10]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[11]  Stephen Grossberg,et al.  On Learning of Spatiotemporal Patterns by Networks with Ordered Sensory and Motor Components 1. Excitatory Components of the Cerebellum , 1969 .

[12]  D L Alkon,et al.  Imaging of memory-specific changes in the distribution of protein kinase C in the hippocampus. , 1989, Science.

[13]  D. Alkon,et al.  Classical conditioning reduces amplitude and duration of calcium-dependent afterhyperpolarization in rabbit hippocampal pyramidal cells. , 1989, Journal of neurophysiology.

[14]  Teuvo Kohonen.  Optical Associative Memories , 1988 .

[15]  A. Coolen,et al.  Image evolution in Hopfield networks. , 1988, Physical review. A, General physics.

[16]  J. Leo van Hemmen,et al.  Statistical Mechanics of Temporal Association in Neural Networks , 1990, NIPS.

[17]  S Dehaene,et al.  Neural networks that learn temporal sequences by selection. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Stephen Grossberg,et al.  Vision, speech, language, and motor control , 1988 .

[19]  S. Grossberg.  Nonlinear difference-differential equations in prediction and learning theory. , 1967, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Mario Markus,et al.  From chemical to biological organization , 1988 .

[21]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[22]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[23]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[24]  J. Ross,et al.  Spectral Kinetics and the Efficiency of (Bio) Chemical Reactions , 1988 .

[25]  Personnaz,et al.  Storage and retrieval of complex sequences in neural networks. , 1988, Physical review. A, General physics.

[26]  S. Grossberg.  Some Networks that can Learn, Remember, and Reproduce any Number of Complicated Space-time , 1970 .

[27]  Wang,et al.  Chaos, multiplicity, crisis, and synchronicity in higher-order neural networks. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[28]  Daniel L. Alkon,et al.  Memory Traces in the Brain , 1987 .

[29]  Kim T. Blackwell,et al.  A new approach to hand-written character recognition , 1992, Pattern Recognit..

[30]  Philip G. Johnson.  Cornell University , 1897, The Journal of comparative medicine and veterinary archives.

[31]  Shiino,et al.  Retrieval of spatio-temporal sequence in asynchronous neural network. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[32]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Second Edition , 1988, Springer Series in Information Sciences.

[33]  S. Grossberg.  The Adaptive Self-Organization of Serial Order in Behavior: Speech, Language, And Motor Control , 1987 .

[34]  John J. Hopfield,et al.  Connected-digit speaker-dependent speech recognition using a neural network with time-delayed connections , 1991, IEEE Trans. Signal Process..

[35]  Lipo Wang,et al.  Physical Modeling of Neural Networks , 1992 .

[36]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[37]  J J Hopfield,et al.  Learning algorithms and probability distributions in feed-forward and feed-back networks. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[38]  P. Peretto,et al.  Collective Properties of Neural Networks , 1986 .

[39]  L. Wang,et al.  Synchronous neural networks of nonlinear threshold elements with hysteresis. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[41]  D Kleinfeld,et al.  Sequential state generation by model neural networks. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[43]  D L Alkon,et al.  Intracellular calcium signals are enhanced for days after Pavlovian conditioning. , 2008, Journal of neurochemistry.

[44]  Eileen C. Schwab,et al.  Pattern recognition by humans and machines , 1986 .

[45]  Richard P. Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[46]  S. Grossberg.  Neural pattern discrimination. , 1970, Journal of theoretical biology.

[47]  L. Wang,et al.  Interactions of neural networks: models for distraction and concentration. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[48]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.