Sparse random networks with LTP learning rules approximate Bayes classifiers via Parzen's method

Abstract Analysis is performed of a neuronal cell layer consisting of sparse random anatomical innervation, incorporating anatomically accurate distribution of excitatory and inhibitory cells, appropriate physiological time courses of excitatory-inhibitory interactions, and synaptic modification via long-term potentiation (LTP). One of two input pathways selects cells that learn and the other uses LTP-based learning rules to modify synapses of selected cells. It is found that the resulting network implements Parzen's method for approximating a Bayes classifier. The simple one-layer network can be generalized to include a class of networks including one that adds an input layer of neurons resulting in changes to the kernel used in the Parzen method. The LTP learning rule is nondecremental and limited to a maximum ceiling, which will result in essential saturation of synaptic weights; nonetheless, calculations using reasonable estimates of network parameters indicate a large network capacity.

[1]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[2]  G. Lynch,et al.  Factors regulating the magnitude of long-term potentiation induced by theta pattern stimulation , 1992, Brain Research.

[3]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[4]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[5]  David Zipser,et al.  The neurobiological significance of the new learning models , 1993 .

[6]  Eytan Domany,et al.  A maximum overlap neural network for pattern recognition , 1987 .

[7]  Teuvo Kohonen,et al.  An Adaptive Associative Memory Principle , 1974, IEEE Transactions on Computers.

[8]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[9]  G. Lynch,et al.  Synapses, circuits, and the beginnings of memory , 1986 .

[10]  F. Macrides,et al.  Olfactory Bulb Units: Activity Correlated with Inhalation Cycles and Odor Quality , 1972, Science.

[11]  G Christofi,et al.  The postsynaptic induction of nonassociative long-term depression of excitatory synaptic transmission in rat hippocampal slices. , 1993, Journal of neurophysiology.

[12]  C. Legg,et al.  An autoradiographic study of the efferent projections of the ventral lateral geniculate nucleus of the hooded rat , 1979, Brain Research.

[13]  G. Lynch,et al.  Derivation of Encoding Characteristics of Layer II Cerebral Cortex , 1989, Journal of Cognitive Neuroscience.

[14]  D. Pandya,et al.  Some connections of the entorhinal (area 28) and perirhinal (area 35) cortices of the rhesus monkey. I. Temporal lobe afferents , 1975, Brain Research.

[15]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[16]  J Ambros-Ingerson,et al.  Simulation of paleocortex performs hierarchical clustering. , 1990, Science.

[17]  D. Marr A theory for cerebral neocortex , 1970, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[18]  Shun-Ichi Amari,et al.  Topographic organization of nerve fields , 1979, Neuroscience Letters.

[19]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[20]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[21]  G. Lynch,et al.  Patterned stimulation at the theta frequency is optimal for the induction of hippocampal long-term potentiation , 1986, Brain Research.

[22]  C. Stevens,et al.  A depression long awaited , 1990, Nature.

[23]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[24]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[25]  R Granger,et al.  Non-Hebbian properties of long-term potentiation enable high-capacity encoding of temporal sequences. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[26]  T. Sejnowski,et al.  Associative long-term depression in the hippocampus induced by hebbian covariance , 1989, Nature.

[27]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[28]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[29]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[30]  W. K. Taylor Cortico-thalamic organization and memory , 1964, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[31]  Deepak N. Pandya,et al.  Some connections of the entorhinal (area 28) and perirhinal (area 35) cortices of the rhesus monkey. II. Frontal lobe afferents , 1975, Brain Research.

[32]  Shigeki Miyake,et al.  A neural network approach to a Bayesian statistical decision problem , 1991, IEEE Trans. Neural Networks.

[33]  Bruce W. Suter,et al.  The multilayer perceptron as an approximation to a Bayes optimal discriminant function , 1990, IEEE Trans. Neural Networks.

[34]  G Lynch,et al.  Brief bursts of high-frequency stimulation produce two types of structural change in rat hippocampus. , 1980, Journal of neurophysiology.

[35]  R. Malenka,et al.  Mechanisms underlying induction of homosynaptic long-term depression in area CA1 of the hippocampus , 1992, Neuron.

[36]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[37]  G. Lynch,et al.  The effects of repetitive low frequency stimulation on control and "potentiated" synaptic responses in the hippocampus. , 1980, Life sciences.

[38]  Patrick A. Shoemaker,et al.  A note on least-squares learning procedures and classification by neural network models , 1991, IEEE Trans. Neural Networks.

[39]  B. Komisaruk,et al.  Synchrony between limbic system theta activity and rhythmical behavior in rats. , 1970, Journal of comparative and physiological psychology.

[40]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[41]  F. Macrides,et al.  Temporal relationships between hippocampal slow waves and exploratory sniffing in hamsters. , 1975, Behavioral biology.

[42]  J M Wyss,et al.  An autoradiographic study of the efferent connections of the entorhinal cortex in the rat , 1981, The Journal of comparative neurology.

[43]  V. Mountcastle,et al.  An organizing principle for cerebral function : the unit module and the distributed system , 1978 .

[44]  Eric A. Wan,et al.  Neural network classification: a Bayesian interpretation , 1990, IEEE Trans. Neural Networks.

[45]  Shigeki Miyake,et al.  Bayes statistical behavior and valid generalization of pattern classifying neural networks , 1991, IEEE Trans. Neural Networks.

[46]  M. Bear,et al.  Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[47]  G. Lynch,et al.  Induction of synaptic potentiation in hippocampus by patterned stimulation involves two events. , 1986, Science.

[48]  Richard Granger,et al.  A cortical model of winner-take-all competition via lateral inhibition , 1992, Neural Networks.

[49]  Yutaka Sakaguchi,et al.  Topographic organization of nerve field with teacher signal , 1990, Neural Networks.

[50]  Gary Lynch,et al.  Reversal of LTP by theta frequency stimulation , 1993, Brain Research.

[51]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[52]  H. Eichenbaum,et al.  Temporal relationship between sniffing and the limbic theta rhythm during odor discrimination reversal learning , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[53]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[54]  T. Kohonen Self-organized formation of topographically correct feature maps , 1982 .

[55]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[56]  Stephen Grossberg,et al.  A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans , 1982 .

[57]  Gary Lynch,et al.  Stable depression of potentiated synaptic responses in the hippocampus with 1–5 Hz stimulation , 1990, Brain Research.

[58]  T. Powell,et al.  The basic uniformity in structure of the neocortex. , 1980, Brain : a journal of neurology.

[59]  T. P. S. Powell,et al.  The callosal connexions of the primary somatic sensory cortex in the monkey , 1985, Brain Research Reviews.