The impact of network topology on self-organizing maps

In this paper, we study instances of complex neural networks, i.e. neural networks with complex topologies. We use Self-Organizing Map neural networks whose neighborhood relationships are defined by a complex network, to classify handwritten digits. We show that topology has a small impact on performance and robustness to neuron failures, at least at long learning times. Performance may however be increased (by almost $10\%$) by evolutionary optimization of the network topology. In our experimental conditions, the evolved networks are more random than their parents, but display a more heterogeneous degree distribution.

[1]  V. Latora,et al.  Complex networks: Structure and dynamics , 2006 .

[2]  Marco Tomassini,et al.  Evolution and Dynamics of Small-World Cellular Automata , 2005, Complex Syst..

[3]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[4]  Guillermo Abramson,et al.  Associative memory on a small-world neural network , 2003, nlin/0310033.

[5]  Michael Menzinger,et al.  Topology and computational performance of attractor neural networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[7]  P. Cluzel,et al.  Effects of topology on network evolution , 2006 .

[8]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[9]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[10]  S. Strogatz.  Exploring complex networks , 2001, Nature.

[11]  Michael Biehl,et al.  Handbook of Brain Theory and Neural Networks (second editon) , 2002 .

[12]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[13]  Ali A. Minai,et al.  Efficient associative memory using small-world architecture , 2001, Neurocomputing.

[14]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[15]  L. da Fontoura Costa,et al.  Efficient Hopfield pattern recognition on a scale-free neural network , 2002, cond-mat/0212601.

[16]  Jon M. Kleinberg,et al.  Navigation in a small world , 2000, Nature.

[17]  Zhidong Deng,et al.  Complex Systems Modeling Using Scale-Free Highly-Clustered Echo State Network , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[18]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[19]  Beom Jun Kim.  Performance of networks of artificial neurons: the role of clustering. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[20]  D. Simard,et al.  Fastest learning in small-world neural networks , 2004, physics/0402076.