Long Short-Term Memory in Echo State Networks: Details of a Simulation Study

Echo State Networks (ESNs) is an approach to design and train recurrent neural networks in supervised learning tasks. An important objective in many such tasks is to learn to exploit long-time dependencies in the processed signals ("long short-term memory" performance). Here we expose ESNs to a series of synthetic benchmark tasks that have been used in the literature to study the learnability of long-range temporal dependencies. This report provides all the detail necessary to replicate these experiments. It is intended to serve as the technical companion to a journal submission paper where the findings are analysed and compared to results obtained elsewhere with other learning paradigms.