Topology influences performance in the associative memory neural networks

Abstract To explore how topology affects performance within Hopfield-type associative memory neural networks (AMNNs), we studied the computational performance of the neural networks with regular lattice, random, small-world, and scale-free structures. In this Letter, we found that the memory performance of neural networks obtained through asynchronous updating from “larger” nodes to “smaller” nodes are better than asynchronous updating in random order, especially for the scale-free topology. The computational performance of associative memory neural networks linked by the above-mentioned network topologies with the same amounts of nodes (neurons) and edges (synapses) were studied respectively. Along with topologies becoming more random and less locally disordered, we will see that the performance of associative memory neural network is quite improved. By comparing, we show that the regular lattice and random network form two extremes in terms of patterns stability and retrievability. For a network, its patterns stability and retrievability can be largely enhanced by adding a random component or some shortcuts to its structured component. According to the conclusions of this Letter, we can design the associative memory neural networks with high performance and minimal interconnect requirements.

[1]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[2]  C. Wu Perturbation of coupling matrices and its effect on the synchronizability in arrays of coupled chaotic systems , 2003, nlin/0307052.

[3]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[4]  Dietrich Stauffer,et al.  Associative recall in non-randomly diluted neuronal networks , 2003, cond-mat/0302040.

[5]  O. Sporns,et al.  Organization, development and function of complex brain networks , 2004, Trends in Cognitive Sciences.

[6]  S. Redner,et al.  Introduction To Percolation Theory , 2018 .

[7]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Jinde Cao,et al.  Exponential stability and periodic oscillatory solution in BAM networks with delays , 2002, IEEE Trans. Neural Networks.

[9]  Béla Bollobás,et al.  Random Graphs , 1985 .

[10]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[11]  Xiao Fan Wang,et al.  Complex Networks: Topology, Dynamics and Synchronization , 2002, Int. J. Bifurc. Chaos.

[12]  Karl E. Kürten,et al.  Quasi-optimized memorization and retrieval dynamics in sparsely connected neural network models , 1990 .

[13]  Reka Albert,et al.  Mean-field theory for scale-free random networks , 1999 .

[14]  V Latora,et al.  Efficient behavior of small-world networks. , 2001, Physical review letters.

[15]  J. Cao,et al.  Periodic oscillatory solution of bidirectional associative memory networks with delays. , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[16]  N Brunel,et al.  Modeling memory: what do we learn from attractor neural networks? , 1998, Comptes rendus de l'Academie des sciences. Serie III, Sciences de la vie.

[17]  Jinde Cao,et al.  Exponential stability of continuous-time and discrete-time bidirectional associative memory networks with delays , 2004 .

[18]  Michael Menzinger,et al.  Topology and computational performance of attractor neural networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[19]  L. da Fontoura Costa,et al.  Efficient Hopfield pattern recognition on a scale-free neural network , 2002, cond-mat/0212601.

[20]  Jon M. Kleinberg,et al.  Navigation in a small world , 2000, Nature.

[21]  Jinde Cao,et al.  Global asymptotic stability of delayed bi-directional associative memory neural networks , 2003, Appl. Math. Comput..

[22]  V. Eguíluz,et al.  Highly clustered scale-free networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  L. da Fontoura Costa,et al.  Growth-driven percolations: the dynamics of connectivity in neuronal systems , 2004, q-bio/0411009.

[24]  M Girvan,et al.  Structure of growing social networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  B. Forrest Vectorized multi-site coding for nearest-neighbour neural networks , 1989 .

[26]  V. Eguíluz,et al.  Growing scale-free networks with small-world behavior. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  A. Barabasi,et al.  Scale-free characteristics of random networks: the topology of the world-wide web , 2000 .

[28]  Leon O. Chua,et al.  Star Cellular Neural Networks for Associative and Dynamic Memories , 2004, Int. J. Bifurc. Chaos.

[29]  Paul J. Flory,et al.  Molecular Size Distribution in Three Dimensional Polymers. I. Gelation1 , 1941 .

[30]  Guanrong Chen,et al.  Complex networks: small-world, scale-free and beyond , 2003 .

[31]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[32]  S. Strogatz Exploring complex networks , 2001, Nature.

[33]  Power–law properties of Chinese stock market , 2005 .

[34]  Paul J. Flory,et al.  Molecular Size Distribution in Three Dimensional Polymers. II. Trifunctional Branching Units , 1941 .

[35]  Guanrong Chen,et al.  A time-varying complex dynamical network model and its controlled synchronization criteria , 2004, IEEE Trans. Autom. Control..

[36]  Reuven Cohen,et al.  Scale-free networks on lattices. , 2002, Physical review letters.