Influence of the neural network topology on the learning dynamics

We study the influence of the topology of a neural network on its learning dynamics. The network topology can be controlled by one parameter p"r"w to convert the topology from regular to random in a continuous way [D.J. Watts and S.H. Strogatz, Collective dynamics of small-world networks, Nature 393 (1998) 440-442]. As test problem, which requires a recurrent network, we choose the problem of timing to be learned by the network, that means to connect a predefined input neuron with a output neuron in exactlyT"f time steps. We analyze the learning dynamics for different parameters numerically by counting the number of paths within the network which are available for solving the problem. Our results show, that there are parameter values for which either a regular, small-world or random network gives the best performance depending strongly on the choice for the predefined input and output neurons.