Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets

According to Hebb's postulate for learning, information presented to a neural net during a learning session is stored in the synaptic efficacies. Long-term potentiation occurs only if the postsynaptic neuron becomes active in a time window set up by the presynaptic one. We carefully interpret and mathematically implement the Hebb rule so as to handle both stationary and dynamic objects such as single patterns and cycles. Since the natural dynamics contains a rather broad distribution of delays, the key idea is to incorporate these delays in the learning session. As theory and numerical simulation show, the resulting procedure is surprisingly robust and faithful. It also turns out that pure Hebbian learning is by selection: the network produces synaptic representations that are selected according to their resonance with the input percepts.

[1]  Robert Miller Representation of brief temporal patterns, Hebbian synapses, and the left-hemisphere dominance for phoneme recognition , 1987, Psychobiology.

[2]  S Dehaene,et al.  Neural networks that learn temporal sequences by selection. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Anthony C. C. Coolen,et al.  Delays in Neural Networks , 1988 .

[4]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[5]  Joachim M. Buhmann,et al.  Noise-driven temporal association in neural networks , 1987 .

[6]  S Dehaene,et al.  Spin glass model of learning by selection. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[7]  S. Grossberg A prediction theory for some nonlinear functional-differential equations. II. Learning of patterns☆ , 1968 .

[8]  Gutfreund,et al.  Processing of temporal sequences in neural networks. , 1988, Physical review letters.

[9]  G. Parisi A memory which forgets , 1986 .

[10]  D J Amit,et al.  Neural networks counting chimes. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Kunihiko Fukushima,et al.  A model of associative memory in the brain , 1973, Kybernetik.

[12]  Conrad,et al.  On the representation of objects and their relations in the brain , 1974 .

[13]  D Kleinfeld,et al.  Sequential state generation by model neural networks. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[15]  W. Little The existence of persistent states in the brain , 1974 .

[16]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[17]  R. Malinow,et al.  Postsynaptic hyperpolarization during conditioning reversibly blocks induction of long-term potentiation , 1986, Nature.

[18]  G. Willwacher Fähigkeiten eines assoziativen Speichersystems im Vergleich zu Gehirnfunktionen , 1976, Biological Cybernetics.

[19]  van Hemmen JL Nonlinear neural networks near saturation. , 1987, Physical review. A, General physics.

[20]  E. Caianiello Outline of a theory of thought-processes and thinking machines. , 1961, Journal of theoretical biology.

[21]  V. Braitenberg Is the cerebellar cortex a biological clock in the millisecond range? , 1967, Progress in brain research.

[22]  J. Hemmen,et al.  Elementary solution of Classical Spin-Glass models , 1986 .

[23]  P. Peretto,et al.  Collective Properties of Neural Networks , 1986 .

[24]  K. H. Lee,et al.  Correlation of cell body size, axon size, and signal conduction velocity for individually labelled dorsal root ganglion cells in the cat , 1986, The Journal of comparative neurology.

[25]  D. Grensing,et al.  Random-site spin-glass models , 1986 .

[26]  Riedel,et al.  Temporal sequences and chaos in neural nets. , 1988, Physical review. A, General physics.

[27]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[28]  Michel Kerszberg,et al.  Synchronization in Neural Assemblies , 1990 .

[29]  V. Braitenberg Two Views of the Cerebral Cortex , 1986 .

[30]  Stanislas Dehaene,et al.  Networks of Formal Neurons and Memory Palimpsests , 1986 .

[31]  Lev B. Ioffe,et al.  The Augmented Models of Associative Memory Asymmetric Interaction and Hierarchy of Patterns - Int. J. Mod. Phys. B1, 51 (1987) , 1987 .

[32]  S. Kelso,et al.  Hebbian synapses in hippocampus. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[33]  J. Hemmen Nonlinear neural networks near saturation. , 1987 .

[34]  B. Gustafsson,et al.  Long-term potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[35]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[36]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[37]  J. L. van Hemmen,et al.  Nonlinear neural networks. II. Information processing , 1988 .

[38]  Gerd Willwacher,et al.  Storage of a temporal pattern sequence in a network , 2004, Biological Cybernetics.

[39]  W. A. Little,et al.  Analytic study of the memory storage capacity of a neural network , 1978 .