Associative memory by neural networks with delays and pruning

In the correlation-type associative memory model it is known that the storage capacity per synapse increases when synapses are pruned. In that case, however, the storage capacity of the whole network is decreased. In order to solve this difficulty, the authors introduce the synapse with delay, and propose a scheme in which the coupling rate of the synapse is reduced while retaining the number of synapses. As an example of such an approach, this paper discusses a discrete-time synchronous model with delayed coupling and synaptic pruning. The dynamics of the model is investigated theoretically by statistical neurodynamics, and the storage capacity is analyzed quantitatively. An analysis is performed for two types of synaptic pruning, random and systematic. In either case, it is seen that the storage capacity increases when the number of delay stages is increased while reducing the synaptic coupling rate, provided that the total number of synapses is kept constant. The increasing tendency is more marked in systematic pruning than in random pruning. These results were verified by comparing the results to those of computer simulation. The significance of the theoretical findings and the excessive generation and pruning of the synapses in the brain are also discussed from the viewpoint of the theory of computation. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(6): 48–58, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10037