Parallel and separable recursive Levenberg-Marquardt training algorithm

A novel decomposed recursive Levenberg Marquardt (RLM) algorithm is derived for the training of feedforward neural networks. By neglecting interneuron weight correlations the recently proposed RLM training algorithm can be decomposed at neuron level enabling weights to be updated in an efficient parallel manner. A separable least squares implementation of decomposed RLM is also introduced. Experiment results for two nonlinear time series problems demonstrate the superiority of the new training algorithms.

[1]  George W. Irwin,et al.  Fast parallel off-line training of multilayer perceptrons , 1997, IEEE Trans. Neural Networks.

[2]  S. Gunnarsson.  Combining tracking and regularization in recursive least squares identification , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[3]  Sheng Chen,et al.  Parallel recursive prediction error algorithm for training layered neural networks , 1990 .

[4]  M. Viberg,et al.  Adaptive neural nets filter using a recursive Levenberg-Marquardt search direction , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[5]  George W. Irwin,et al.  A hybrid linear/nonlinear training algorithm for feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[6]  Vijanth S. Asirvadam,et al.  Separable recursive training algorithms for feedforward neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[7]  Patrick van der Smagt.  Minimisation methods for training feedforward neural networks , 1994, Neural Networks.

[8]  Stephen A. Billings,et al.  A comparison of the backpropagation and recursive prediction error algorithms for training neural networks , 1991 .

[9]  Jonas Sjöberg,et al.  Efficient training of neural nets for nonlinear adaptive filtering using a recursive Levenberg-Marquardt algorithm , 2000, IEEE Trans. Signal Process..

[10]  Sheng Chen,et al.  Recursive prediction error parameter estimator for non-linear models , 1989 .

[11]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .