Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down

A recent article (Stanton and Sejnowski 1989) on long-term synaptic depression in the hippocampus has reopened the issue of the computational efficiency of particular synaptic learning rules (Hebb 1949; Palm 1988a; Morris and Willshaw 1989) homosynaptic versus heterosynaptic and monotonic versus nonmonotonic changes in synaptic efficacy. We have addressed these questions by calculating and maximizing the signal-to-noise ratio, a measure of the potential fidelity of recall, in a class of associative matrix memories. Up to a multiplicative constant, there are three optimal rules, each providing for synaptic depression such that positive and negative changes in synaptic efficacy balance out. For one rule, which is found to be the Stent-Singer rule (Stent 1973; Rauschecker and Singer 1979), the depression is purely heterosynaptic; for another (Stanton and Sejnowski 1989), the depression is purely homosynaptic; for the third, which is a generalization of the first two, and has a higher signal-to-noise ratio, it is both heterosynaptic and homosynaptic. The third rule takes the form of a covariance rule (Sejnowski 1977a,b) and includes, as a special case, the prescription due to Hopfield (1982) and others (Willshaw 1971; Kohonen 1972).

[1]  Professor Dr. John C. Eccles,et al.  The Cerebellum as a Neuronal Machine , 1967, Springer Berlin Heidelberg.

[2]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[3]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[4]  David Willshaw,et al.  Models of distributed associative memory , 1971 .

[5]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[6]  G. Stent.  A physiological mechanism for Hebb's postulate of learning. , 1973, Proceedings of the National Academy of Sciences of the United States of America.

[7]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[8]  T. Sejnowski.  Statistical constraints on synaptic plasticity. , 1977, Journal of theoretical biology.

[9]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[10]  W. Singer,et al.  Changes in the circuitry of the kitten visual cortex are gated by postsynaptic activity , 1979, Nature.

[11]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[12]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  G. Collingridge,et al.  Excitatory amino acids in synaptic transmission in the Schaffer collateral‐commissural pathway of the rat hippocampus. , 1983, The Journal of physiology.

[14]  R. Linsker,et al.  From basic network principles to neural architecture , 1986 .

[15]  G. Lynch,et al.  Selective impairment of learning and blockade of long-term potentiation by an N-methyl-D-aspartate receptor antagonist, AP5 , 1986, Nature.

[16]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[17]  E. Gardner,et al.  Maximum Storage Capacity in Neural Networks , 1987 .

[18]  Sompolinsky,et al.  Information storage in neural networks with low levels of activity. , 1987, Physical review. A, General physics.

[19]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[20]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[21]  Günther Palm,et al.  On the asymptotic information storage capacity of neural networks , 1988 .

[22]  Günther Palm,et al.  Local Synaptic Rules with Maximal Information Storage Capacity , 1988 .

[23]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[24]  T. Sejnowski,et al.  Associative long-term depression in the hippocampus induced by hebbian covariance , 1989, Nature.

[25]  T. Sejnowski,et al.  Induction of synaptic plasticity by Hebbian covariance in the Hippocampus , 1989 .

[26]  D. Willshaw,et al.  Must what goes up come down? , 1989, Nature.

[27]  T. Sejnowski.  Synaptic mechanisms for long-term depression , 1991, Current Biology.

[28]  William A. Phillips,et al.  A Biologically Supported Error-Correcting Learning Rule , 1991, Neural Computation.