Recurrent neural network with short-term memory and fast structural learning method

The Elman network is used frequently in learning time series. However, it has several problems: (1) a comparatively long learning time, (2) learning is not necessarily successful, (3) lack of discussions on a generalization ability, and (4) the necessity of deciding unit numbers and the like by trial and error. In this study, a network providing short-term memory for the Elman network in order to facilitate learning of time series is proposed. In addition, an algorithm considering both structural learning and fast learning, which have been regarded as having a trade-off relationship in the past, is proposed as a learning algorithm. It has been verified by tests that the learning time, convergence rate, and generalization ability are improved by the proposed method. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(6): 69–79, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.1206