An Improved Algorithm for Eleman Neural Network by Adding a Modified Error Function

The Eleman Neural Network has been widely used in various fields ranging from temporal version of the Exclusive-OR function to the discovery of syntactic categories in natural language date. However, one of the problems often associated with this type of network is the local minima problem which usually occurs in the process of the learning. To solve this problem, we have proposed an error function which can harmonize the update weights connected to the hidden layer and those connected to the output layer by adding one term to the conventional error function. It can avoid the local minima problem caused by this disharmony. We applied this method to the Boolean Series Prediction Questions problems to demonstrate its validity. The result shows that the proposed method can avoid the local minima problem and largely accelerate the speed of the convergence and get good results for the prediction tasks.

[1]  H. Tamura,et al.  An improved backpropagation algorithm to avoid the local minima problem , 2004, Neurocomputing.

[2]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[3]  Duc Truong Pham,et al.  Identification of linear and nonlinear dynamic systems using recurrent neural networks , 1993, Artif. Intell. Eng..

[4]  C. Lee Giles,et al.  Extraction of rules from discrete-time recurrent neural networks , 1996, Neural Networks.

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  Seref Sagiroglu,et al.  Elman Network with Embedded Memory for System Identification , 2006, J. Inf. Sci. Eng..

[7]  Michael I. Jordan Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[8]  Mekki Ksouri,et al.  A comparative study on the channel modeling using feedforward and recurrent neural network structures , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[9]  J D Cohen,et al.  A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. , 1990, Science.

[10]  Seppo J. Ovaska,et al.  A modified Elman neural network model with application to dynamical systems identification , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).