Augmentation of Elman Recurrent Network Learning with Particle Swarm Optimization

Despite a variety of artificial neural network (ANN) categories, backpropagation network (BP) and Elman recurrent network (ERN) are the widespread modus operandi in real applications. However, there are many drawbacks in BP network, for instance, confinement in finding local minimum and may get stuck at regions of a search space or trap in local minima. To solve these problems, various optimization techniques such as particle swarm optimization (PSO) and genetic algorithm (GA) have been executed to improve ANN performance. In this study, we exploit errors optimization of Elman recurrent network with backpropagation (ERNBP) and Elman recurrent network with particle swarm optimization (ERNPSO) to probe the performance of both networks. The comparisons are done with PSO that is integrated with neural network (PSONN) and GA with neural network (GANN). The results show that ERNPSO furnishes promising outcomes in terms of classification accuracy and convergence rate compared to ERNBP, PSONN and GANN.

[1]  Andries Petrus Engelbrecht,et al.  Cooperative learning in neural networks using particle swarm optimizers , 2000, South Afr. Comput. J..

[2]  Peter J. Bentley,et al.  Particle swarm optimization recommender system , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[3]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[4]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[5]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[6]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[7]  J. Dayhoff,et al.  Artificial neural networks , 2001, Cancer.

[8]  B. Yegnanarayana,et al.  Artificial Neural Networks , 2004 .

[9]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[10]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[11]  A. Roli Artificial Neural Networks , 2012, Lecture Notes in Computer Science.

[12]  Frans van den Bergh,et al.  Particle Swarm Weight Initialization In Multi-Layer Perceptron Artificial Neural Networks , 1999 .