Perspectives and challenges for recurrent neural network training

Recurrent neural networks (RNNs) offer flexible machine learning tools which share the learning abilities of feedforward networks and which extend their expression abilities based on dynamical equations. Hence, they can directly process complex spatiotemporal data and model complex dynamic systems. Since temporal and spatial data are present in many domains such as processing environmental time series, modelling the financial market, speech and language processing, robotics, bioinformatics, medical informatics, etc., RNNs constitute promising candidates for a variety of applications. Further, their rich dynamic repertoire as time dependent systems makes them suitable candidates for modelling brain phenomena or mimicking large-scale distributed computations and argumentations. Thus, RNNs carry the promise of efficient biologically plausible signal processing models optimally suited for a wide area of industrial applications on the one hand and an explanation of cognitive phenomena of the human brain on the other hand. Despite these facts, however, the design of efficient training methods for RNNs as well as their mathematical investigation with respect to reliable information representation and generalization ability when dealing with complex data structures is still a challenge. It has led to diverse approaches and architectures including specific training modes such as echo and liquid-state-machines, backpropagation decorrelation, or long short term memory, specific architectures such as recursive and graph networks, and hybrid systems at the borderline of symbolic and subsymbolic processing such as the core method. Interestingly, very heterogeneous domains are included and contributions to the area of RNNs stem from very different fields of research including, for example, logic, iterated function systems, and biological networks. The aim of this special issue is to collect recent work developed in the field of recurrent information processing, which bridges the gap between different approaches and which sheds some light on canonical solutions or principled problems which occur in the context of recursive information processing when considered across the disciplines. This idea was born during a Dagstuhl seminar entitled ‘Recurrent Neural NetworksModels, Capacities, and Applications’ which took place in 2008 and which centered around the connection of RNNs to biological systems on the one side and logical models on the other side, gathering together experts in all three domains. This volume contains five papers which were accepted out of