Cross-talk theory of memory capacity in neural networks

The present paper presents a theory for the mechanics of cross-talk among constituent neurons in networks in which multiple memory traces have been embedded, and develops criteria for memory capacity based on the disruptive influences of this cross-talk. The theory is based on interconnection patterns defined by the sequential configuration model of dynamic firing patterns. The theory accurately predicts the memory capacities observed in computer simulated nets, and predicts that cortical-like modules should be able to store up to about 300–900 selectively retrievable memory traces before disruption by cross-talk is likely. It also predicts that the cortex may has designed itself for modules of 30,000 neurons to at least in part to optimize memory capacity.

[1]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[2]  J. Szentágothai.  The modular architectonic principle of neural centers. , 1983, Reviews of physiology, biochemistry and pharmacology.

[3]  James L. McClelland,et al.  Open Questions About Computation in Cerebral Cortex , 1987 .

[4]  V. Braitenberg.  Cell Assemblies in the Cerebral Cortex , 1978 .

[5]  Professor Dr. Valentino Braitenberg,et al.  On the Texture of Brains , 1977, Heidelberg Science Library.

[6]  Professor Moshe Abeles,et al.  Local Cortical Circuits , 1982, Studies of Brain Function.

[7]  J. Eccles.  The modular operation of the cerebral neocortex considered as the material basis of mental events , 1981, Neuroscience.

[8]  Günther Palm,et al.  Local rules for synaptic modification in neural networks , 1993 .

[9]  R. J. MacGregor,et al.  Sequential configuration model for firing patterns in local neural networks , 1991, Biological Cybernetics.

[10]  A. Aertsen,et al.  Neuronal assemblies , 1989, IEEE Transactions on Biomedical Engineering.

[11]  Günther Palm.  Rules for synaptic changes and their relevance for the storage of information in the brain , 1982 .

[12]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[13]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[14]  Shun-ichi Amari,et al.  Characteristics of sparsely encoded associative memory , 1989, Neural Networks.

[15]  G. Palm.  On the storage capacity of an associative memory with randomly distributed storage elements , 2004, Biological Cybernetics.

[16]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[17]  Ronald J. MacGregor,et al.  Neural and brain modeling , 1987 .

[18]  M. Abeles,et al.  Neuronal activities related to higher brain functions-theoretical and experimental implications , 1989, IEEE Transactions on Biomedical Engineering.

[19]  Günther Palm,et al.  On the asymptotic information storage capacity of neural networks , 1988 .