Structure Optimization Using Adaptive Particle Swarm Optimization

Abstract In this paper a method to optimize the structure of neural network named as Adaptive Particle Swarm Optimization (PSO) has been proposed. In this method nested PSO has been used. Each particle in outer PSO is used for different network construction. The particles update themselves in each iteration by following the global best and personal best performances. The inner PSO is used for training the networks and evaluate the performance of the networks. The effectiveness of this method is tested on many benchmark datasets to find out their optimum structure and the results are compared with other population based methods and finally it is implemented on classification using neural network in data mining.

[1]  Huihe Shao,et al.  An ANN's evolved by a new evolutionary system and its application , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[2]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[3]  Yung C. Shin,et al.  Constructive training of recurrent neural networks using hybrid optimization , 2010, Neurocomputing.

[4]  Cheng-Jian Lin,et al.  A self-adaptive quantum radial basis function network for classification applications , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[5]  Tung-Kuan Liu,et al.  Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm , 2006, IEEE Trans. Neural Networks.

[6]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[7]  Michael K. Weir,et al.  A method for self-determination of adaptive learning rates in back propagation , 1991, Neural Networks.

[8]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[9]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[10]  M.K. Madrid,et al.  A simple adaptive back-propagation algorithm for multilayered feedforward perceptrons , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[11]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[12]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[13]  Zne-Jung Lee,et al.  Parameter determination of support vector machine and feature selection using simulated annealing approach , 2008, Appl. Soft Comput..

[14]  De-Shuang Huang,et al.  A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks , 2008, IEEE Transactions on Neural Networks.

[15]  B. B. Misra,et al.  Functional Link Artificial Neural Network for Classification Task in Data Mining , 2007 .

[16]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[17]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[18]  Alireza Sadeghian,et al.  A Variation of Particle Swarm Optimization for Training of Artificial Neural Networks , 2010 .

[19]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[20]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.