Using a genetic algorithm to investigate efficient connectivity in associative memories

We investigate sparse networks of threshold units, trained with the perceptron learning rule to act as associative memories. The units have position and are placed in a ring so that the wiring cost is a meaningful measure. A genetic algorithm is used to evolve networks that have efficient wiring, but also good functionality. It is shown that this is possible, and that the connection strategy used by the networks appears to maintain some connectivity at all distances, but with the probability of a connection decreasing rapidly with distance.

[1]  Amir Ayali,et al.  Morphological characterization of in vitro neuronal networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  A. Barabasi,et al.  Scale-free characteristics of random networks: the topology of the world-wide web , 2000 .

[3]  Neil Davey,et al.  Efficient architectures for sparsely-connected high capacity associative memory models , 2007, Connect. Sci..

[4]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[5]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[6]  M. Brucoli,et al.  Discrete-time cellular neural networks for associative memories: a new design method via iterative learning and forgetting algorithms , 1995, 38th Midwest Symposium on Circuits and Systems. Proceedings.

[7]  L. da Fontoura Costa,et al.  Efficient Hopfield pattern recognition on a scale-free neural network , 2002, cond-mat/0212601.

[8]  N. Davey,et al.  High capacity associative memories and small world networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[9]  Olaf Sporns,et al.  The small world of the cerebral cortex , 2007, Neuroinformatics.

[10]  Neil Davey,et al.  Gaussian and exponential architectures in small-world associative memories , 2006, ESANN.

[11]  Raul Rodriguez-Esteban,et al.  Global optimization of cerebral cortex layout. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Mark Newman,et al.  Models of the Small World , 2000 .

[13]  Joaquín J. Torres,et al.  Influence of topology on the performance of a neural network , 2004, Neurocomputing.

[14]  Michael Menzinger,et al.  Topology and computational performance of attractor neural networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Marcus Kaiser,et al.  Clustered organization of cortical connectivity , 2007, Neuroinformatics.

[16]  Jan Karbowski.  Optimal wiring in the cortex and neuronal degree of separation , 2002, Neurocomputing.

[17]  Neil Davey,et al.  High capacity associative memories and connection constraints , 2004, Connect. Sci..

[18]  Prof. Dr. Dr. Valentino Braitenberg,et al.  Cortex: Statistics and Geometry of Neuronal Connectivity , 1998, Springer Berlin Heidelberg.

[19]  G. Cecchi,et al.  Scale-free brain functional networks. , 2003, Physical review letters.

[20]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[21]  Bernhard Hellwig,et al.  A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex , 2000, Biological Cybernetics.

[22]  Noest.  Domains in neural networks with restricted-range interactions. , 1989, Physical review letters.

[23]  C Cherniak,et al.  Component placement optimization in the brain , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[25]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[26]  Terrence J Sejnowski,et al.  Communication in Neuronal Networks , 2003, Science.

[27]  Ali A. Minai,et al.  Efficient associative memory using small-world architecture , 2001, Neurocomputing.

[28]  O. Sporns,et al.  Organization, development and function of complex brain networks , 2004, Trends in Cognitive Sciences.

[29]  G. Buzsáki,et al.  Interneuron Diversity series: Circuit complexity and axon wiring economy of cortical interneurons , 2004, Trends in Neurosciences.

[30]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[31]  Reka Albert,et al.  Mean-field theory for scale-free random networks , 1999 .

[32]  Neil Davey,et al.  High Capacity Recurrent Associative Memories , 2002, Neurocomputing.

[33]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Neil Davey,et al.  Connectivity graphs and the performance of sparse associative memory models , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[35]  Evelyn Fox Keller,et al.  Revisiting "scale-free" networks. , 2005, BioEssays : news and reviews in molecular, cellular and developmental biology.