Low firing rates: an effective Hamiltonian for excitatory neurons

The authors analyse the behaviour of an attractor neural network which exhibits low mean temporal activity levels, despite the fact that the intrinsic neuronal cycle time is very short (2-3 ms). Information and computation are represented on the excitatory neurons only. The influence of inhibitory neurons, which are assumed to react on a shorter timescale than the excitatory ones, is expressed as an effective interaction of the excitatory neurons. This leads to an effective model, which describes the interplay of excitation and inhibition acting on excitatory neurons in terms of the excitatory neural variables alone. The network operates in the presence of fast noise, which is large relative to the frozen randomness induced by the stored patterns. The overall fraction of active neurons is controlled by a single free parameter, which expresses the relative strength of the effective inhibition. Associative retrieval is identified, as usual, with the breakdown of ergodicity in the dynamics of the network, in particular with the presence of dynamical attractors corresponding to the retrieval of a given pattern. In such an attractor, the activity of neurons corresponding to active sites in the stored patterns increases at the expense of other neurons. Yet only a small fraction of the neurons active in the pattern are in the active state in each elementary time cycle, and they vary from cycle to cycle in an uncorrelated fashion, due to the noise. Hence, the observed mean activity rate of any individual neuron is kept low. This scenario is demonstrated by an analytical study based on the replica method, and the results are tested by numerical simulations.

[1]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[2]  A Treves,et al.  Associative memory neural network with low temporal spiking rates. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[3]  M Sur,et al.  Modular distribution of neurons with slowly adapting and rapidly adapting responses in area 3b of somatosensory cortex in monkeys. , 1984, Journal of neurophysiology.

[4]  Idan Segev,et al.  Synaptic integration mechanisms. Theoretical and experimental investigation of temporal postsynaptic interactions between excitatory and inhibitory inputs. , 1983, Biophysical journal.

[5]  B. Katz,et al.  The effect of inhibitory nerve impulses on a crustacean muscle fibre , 1953, The Journal of physiology.

[6]  Sompolinsky,et al.  Information storage in neural networks with low levels of activity. , 1987, Physical review. A, General physics.

[7]  Professor Moshe Abeles,et al.  Local Cortical Circuits , 1982, Studies of Brain Function.

[8]  Nava Rubin,et al.  Neural networks with low local firing rates , 1989 .

[9]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[10]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[11]  R. Glauber.  Time‐Dependent Statistics of the Ising Model , 1963 .

[12]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[13]  R. Andersen,et al.  The influence of the angle of gaze upon the excitability of the light- sensitive neurons of the posterior parietal cortex , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  S. Kirkpatrick,et al.  Infinite-ranged models of spin-glasses , 1978 .

[15]  Y. Miyashita,et al.  Neuronal correlate of pictorial short-term memory in the primate temporal cortexYasushi Miyashita , 1988, Nature.

[16]  C. Bruce,et al.  Cerebral cortical activity associated with the orientation of visual attention in the rhesus monkey , 1985, Vision Research.

[17]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[18]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[19]  B. L. Ginsborg.  THE PHYSIOLOGY OF SYNAPSES , 1964 .

[20]  S. Shinomoto,et al.  A cognitive and associative memory , 2004, Biological Cybernetics.

[21]  J. Buhmann,et al.  Associative memory with high information content. , 1989, Physical review. A, General physics.

[22]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[23]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .