Successive approximation training algorithm for feedforward neural networks

Abstract A novel algorithm based on successive approximation training for feedforward neural networks is presented in this paper. The convergence of the algorithm is analysed theoretically and the training error is estimated. Theoretical analysis shows that the novel training algorithm is able to overcome the stalemate problem in the later training stage of the traditional algorithms. Numerical experiments show that the proposed algorithm increases the rate of convergence and improves the generalization performance by avoiding local minima.

[1]  王政,et al.  A STUDY OF IDENTIFICATION OF NONLINEAR CHARACTERISTICS IN CUSHIONING PACKAGING BASED ON NEURAL NETWORKS , 1997 .

[2]  Richard J. Mammone,et al.  Artificial neural networks for speech and vision , 1994 .

[3]  Yanchun Liang,et al.  IDENTIFICATION OF RESTORING FORCES IN NON-LINEAR VIBRATION SYSTEMS BASED ON NEURAL NETWORKS , 1997 .

[4]  Chuanyi Ji,et al.  An Efficient EM-based Training Algorithm for Feedforward Neural Networks , 1997, Neural Networks.

[5]  L. Cooper,et al.  When Networks Disagree: Ensemble Methods for Hybrid Neural Networks , 1992 .

[6]  Yanchun Liang,et al.  Advances in identification of nonlinear characteristics of packaging based on computational intelligence , 2000 .

[7]  Andrew Luk,et al.  Fast Convergent Generalized Back-Propagation Algorithm with Constant Learning Rate , 2004, Neural Processing Letters.

[8]  Yanchun Liang,et al.  Application of neural networks to identification of nonlinear characteristics in cushioning packaging , 1996 .

[9]  Yanchun Liang,et al.  Identification of non-linearities in cushioning packaging using neural networks with fuzzy adaptive control , 1997 .

[10]  Brijesh Verma,et al.  Fast training of multilayer perceptrons , 1997, IEEE Trans. Neural Networks.

[11]  Zhao Mingsheng,et al.  A new kind of neuron model with a tunable activation function and its applications , 1997 .

[12]  Chunguang Zhou,et al.  Identification of nonlinear characteristics in cushioning packaging using genetic evolutionary neural networks , 1998 .

[13]  Kyu Ho Park,et al.  Fast learning method for back-propagation neural network by evolutionary adaptation of learning rates , 1996, Neurocomputing.

[14]  R. Hecht-Nielsen,et al.  Theory of the Back Propagation Neural Network , 1989 .