Training Synaptic Delays in a Recurrent Neural Network

Algorithms for training recurrent neural networks (RNN), both with and without time delays, to perform certain tasks are notorious as a major computer resources consumers. These algorithms adapt a set of network parameters for performing certain tasks typical for the network's architecture. The addition of the possibility for synaptic delays modiication enlarges the parameter space of the network, thus making the training task even more demanding and the network more sensitive to the choice of parameters. The main incentives for adding this possibility are rooted in our knowledge about biological nets as well as in practical and applicable reasons, ranging from enhancing generalization to enhancing network capabilities. In this work we show how addition of time delay parameters results in incrementing both network capacity and capabilities and enabling the use of smaller and more competent nets for the same tasks. In addition, the current work introduces several methods to accomplish recurrent neural networks training, optimizing both the synaptic time delays and the weights. Several training methods are evaluated. The most eecient one, based on adaptive simulated an-ii nealing, enables training of recurrent neural networks with synaptic time delays even on a low-end (Personal Computer) platform. The performance of the various training methods and the capabilities of recurrent neural networks with time delays as a whole are examined via extensive computer simulations using typical benchmark tasks. In addition, special new conngurations of recurrent neural networks with time delays, modiications of the general connguration, are presented and examined together with special training algorithms.