A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm

In this letter, a new approach for the learning process of multilayer feedforward neural network is introduced. This approach minimizes a modified form of the criterion used in the standard backpropagation algorithm. This criterion is based on the sum of the linear and the nonlinear quadratic errors of the output neuron. The quadratic linear error signal is appropriately weighted. The choice of the weighted design parameter is evaluated via rank convergence series analysis and asymptotic constant error values. The new proposed modified standard backpropagation algorithm (MBP) is first derived on a single neuron-based net and then extended to a general feedforward neural network. Simulation results of the 4-b parity checker and the circle in the square problem confirm that the performance of the MBP algorithm exceed the standard backpropagation (SBP) in the reduction of the total number of iterations and in the learning time.

[1]  Mohamed Najim,et al.  Evaluation of the feedforward neural network covariance matrix error , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[2]  R. Serfling.  Approximation Theorems of Mathematical Statistics , 1980 .

[3]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[4]  Frank Bärmann,et al.  A learning algorithm for multilayered neural networks based on linear least squares problems , 1993, Neural Networks.

[5]  Daniel Graupe,et al.  Time series analysis: Identification and adaptive filtering , 1984 .

[6]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[7]  S. Citrin,et al.  Fast learning process of multi-layer neural nets using recursive least squares technique , 1989, International 1989 Joint Conference on Neural Networks.

[8]  Chih-Cheng Chen,et al.  A fast multilayer neural-network training algorithm based on the layer-by-layer optimizing procedures , 1996, IEEE Trans. Neural Networks.

[9]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[10]  M.R. Azimi-Sadjadi,et al.  Supervised learning process of multi-layer perceptron neural networks using fast least squares , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[11]  Nazif Tepedelenlioglu,et al.  A fast new algorithm for training feedforward neural networks , 1992, IEEE Trans. Signal Process..

[12]  Anastasios N. Venetsanopoulos,et al.  Fast learning algorithms for neural networks , 1992 .

[13]  D.R. Hush,et al.  Progress in supervised neural networks , 1993, IEEE Signal Processing Magazine.

[14]  H. Keller,et al.  Analysis of Numerical Methods , 1969 .

[15]  Mahmood R. Azimi-Sadjadi,et al.  Fast learning process of multilayer neural networks using recursive least squares method , 1992, IEEE Trans. Signal Process..

[16]  Mohamed Najim,et al.  A new fast neural network training algorithm based on a modified standard backpropagation (MBP) criterion minimization , 1999, NSIP.

[17]  Rudy Setiono,et al.  Use of a quasi-Newton method in a feedforward neural network construction algorithm , 1995, IEEE Trans. Neural Networks.