论文信息 - Stability of backpropagation-decorrelation efficient O(N) recurrent learning

Stability of backpropagation-decorrelation efficient O(N) recurrent learning

We provide a stability analysis based on nonlinear feedback theory for the recently introduced backpropagation-decorrelation (BPDC) recurrent learning algorithm. For one output neuron BPDC adapts only the output weights of a possibly large network and therefore can learn in O(N). We derive a simple sufficient stability inequality which can easily be evaluated and monitored online to assure that the recurrent network remains stable while adapting. As byproduct we show that BPDC is highly competitive on the recently introduced CATS benchmark data (1).

Jochen J. Steil

[1] D. Wunsch,et al. Time series prediction with recurrent neural networks using a hybrid PSO-EA algorithm , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2] Jouko Lampinen,et al. Time series prediction by Kalman smoother with cross-validated noise density , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[3] Jochen J. Steil,et al. Perspectives on learning with recurrent neural networks , 2002, ESANN.

[4] Jochen J. Steil,et al. Input output stability of recurrent neural networks , 1999 .

[5] C. A. Desoer,et al. Nonlinear Systems Analysis , 1978 .

[6] Jochen J. Steil,et al. Analyzing the weight dynamics of recurrent learning algorithms , 2005, Neurocomputing.

[7] Amir F. Atiya,et al. New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[8] Herbert Jaeger,et al. Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[9] Henry Markram,et al. The "Liquid Computer": A Novel Strategy for Real-Time Computing on Time Series , 2002 .