A unifying view of some training algorithms for multilayer perceptrons with FIR filter synapses

Concerns neural network architectures for modelling time-dependent signals. A number of algorithms have been published for multilayer perceptrons with synapses described by finite impulse response (FIR) and infinite impulse response (IIR) filters (the latter case is also known as locally recurrent globally feedforward networks). The derivations of these algorithms have used different approaches in calculating the gradients, and in this paper we present a short, but unifying account of how these different algorithms compare for the FIR case, both in derivation, and performance. A new algorithm is subsequently presented. In this paper, results are compared for the Mackey-Glass chaotic time series (1977) against a number of other methods including a standard multilayer perceptron, and a local approximation method.<<ETX>>

[1]  Ah Chung Tsoi,et al.  An Adaptive Lattice Architecture for Dynamic Multilayer Perceptrons , 1992, Neural Computation.

[2]  Eric A. Wan,et al.  Finite impulse response neural networks with applications in time series prediction , 1994 .

[3]  M. Casdagli.  Chaos and Deterministic Versus Stochastic Non‐Linear Modelling , 1992 .

[4]  M. Gori,et al.  BPS: a learning algorithm for capturing the dynamic nature of speech , 1989, International 1989 Joint Conference on Neural Networks.

[5]  Andreas S. Weigend,et al.  The Future of Time Series: Learning and Understanding , 1993 .

[6]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[7]  Giovanni Soda,et al.  Local Feedback Multilayered Networks , 1992, Neural Computation.

[8]  James P. Crutchfield,et al.  Geometry from a Time Series , 1980 .

[9]  R. R. Leighton,et al.  The autoregressive backpropagation algorithm , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[10]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[11]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[12]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .

[13]  Ah Chung Tsoi,et al.  FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling , 1991, Neural Computation.

[14]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[15]  Eric A. Wan,et al.  Temporal backpropagation for FIR neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[16]  Yoshua Bengio,et al.  Learning the dynamic nature of speech with back-propagation for sequences , 1992, Pattern Recognit. Lett..

[17]  Neal B. Abraham,et al.  Lorenz-like chaos in NH3-FIR lasers , 1995 .