Particle Swarm Optimization For Neural Network Learning Enhancement

Algoritma rambatan balik telah digunakan secara meluas dalam menyelesaikan pelbagai masalah dengan menggunakan konsep perceptron multi aras. Namun begitu terdapat banyak isu utama pada algoritma ini seperti kadar penumpuan yang lambat dan kekerapan terperangkap di dalam minimum setempat. Bagi mengatasi masalah ini, Algoritma Genetik (AG) digunakan untuk menentukan nilai yang optimal bagi mendapatkan parameter yang sesuai seperti kadar pembelajaran dan kadar momentum serta pengoptimuman pemberat. Walaupun AG berjaya meningkatkan keupayaan pembelajaran bagi Rangkaian Neural (RN) menggunakan rambatan balik, masih terdapat beberapa masalah lain seperti latihan untuk mengeluarkan output mengambil masa yang lama dan penggunaan fungsi yang rumit seperti perhitungan pilihan, silangan dan mutasi. Kajian ini mengemukakan teknik pengoptimuman yang terkini, iaitu pengoptimuman partikel secara berkumpulan yang dicerap di dalam proses pembelajaran RN, bagi meningkatkan masa penumpuan dan ketepatan pengelasan. Dua uji kaji telah dilaksanakan, iaitu RN ke hadapan menggunakan pengoptimuman partikel secara berkumpulan dan RN rambatan balik menggunakan AG. Hasil kajian mendapati bahawa RN ke hadapan dengan pengoptimuman partikel secara berkumpulan memberikan keputusan yang lebih baik dari aspek masa penumpuan dan ketepatan pengelasan, berbanding dengan RN rambatan balik menggunakan AG. Kata kunci: Partikel berkumpulan; rangkaian neural; algoritma genetik; rambatan balik; kepintaran berkumpulan Backpropagation (BP) algorithm is widely used to solve many real world problems by using the concept of Multilayer Perceptron (MLP). However, major disadvantages of BP are its convergence rate is relatively slow and always being trapped at the local minima. To overcome this problem, Genetic Algorithm (GA) has been used to determine optimal value for BP parameters such as learning and momentum rate and, also for weight optimization. Although GA has successfully improved Backpropagation Neural Network (BPNN) learning, there are still some issues such as longer training time to produce the output and usage of complex functions in selection, crossover and mutation calculation. In this study, Particle Swarm Optimization (PSO) algorithm has been chosen and applied in feedforward neural network to enhance the learning process in terms of convergence rate and classification accuracy. Two experiments have been conducted; Particle Swarm Optimization Feedforward Neural Network (PSONN) and Genetic Algorithm Backpropagation Neural Network (GANN). The results show that PSONN give promising results in terms of convergence rate and classification accuracy compared to GANN. Key words: Particle swarm; neural network; genetic algorithm; backpropagation; swarm intelligence

[1]  C.K. Mohan,et al.  Training feedforward neural networks using multi-phase particle swarm optimization , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[2]  Victor Ciesielski,et al.  Using Back Propagation Algorithm and Genetic Algorithm to Train and Refine Neural Networks for Object Detection , 1999, DEXA.

[3]  M. J. Paulik,et al.  Multi-layer neural network classification of on-line signatures , 1996, Proceedings of the 39th Midwest Symposium on Circuits and Systems.

[4]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[5]  Cathy H. Wu,et al.  Protein classification using a neural network database system , 1991, ANNA '91.

[6]  M. Cheriet,et al.  A comparative study of fast neural network learning algorithms , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[7]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Yahya H. Zweiri,et al.  A three-term backpropagation algorithm , 2003, Neurocomputing.

[10]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[11]  Andries Petrus Engelbrecht,et al.  Cooperative learning in neural networks using particle swarm optimizers , 2000, South Afr. Comput. J..

[12]  M. Tim Jones,et al.  AI Application Programming , 2003 .

[13]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[14]  Barbara Webb,et al.  Swarm Intelligence: From Natural to Artificial Systems , 2002, Connect. Sci..

[15]  Muhammad Moinuddin,et al.  Automatic classification of speech and music using neural networks , 2004, MMDB '04.

[16]  Frans van den Bergh,et al.  Particle Swarm Weight Initialization In Multi-Layer Perceptron Artificial Neural Networks , 1999 .

[17]  George F. Luger,et al.  Artificial intelligence - structures and strategies for complex problem solving (2. ed.) , 1993 .

[18]  Byung-Ha Ahn,et al.  A Comparison of GA and PSO for Excess Return Evaluation in Stock Markets , 2005, IWINAC.

[19]  Jessica Lowell.  Neural Network , 2001 .

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Richard W. Conners,et al.  Fast Back-Propagation Learning Using Steep Activation Functions and Automatic Weight , 1992 .

[22]  Simon M. Tam,et al.  Multi-layer neural network , 1991 .

[23]  Jonathan M. Roberts,et al.  Flight control using an artificial neural network , 2000 .

[24]  Randall S. Sexton,et al.  Data mining using a genetic algorithm-trained neural network , 2001, Intell. Syst. Account. Finance Manag..

[25]  Yu Li,et al.  Particle swarm optimisation for evolving artificial neural network , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[26]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[27]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[28]  Guo-Chang Gu,et al.  Research on particle swarm optimization: a review , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[29]  Tai-Hoon Cho,et al.  Fast backpropagation learning using steep activation functions and automatic weight reinitialization , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[30]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[31]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[32]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[33]  W. Charytoniuk,et al.  Neural network design for short-term load forecasting , 2000, DRPT2000. International Conference on Electric Utility Deregulation and Restructuring and Power Technologies. Proceedings (Cat. No.00EX382).

[34]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).