A hybrid particle swarm optimization and its application in neural networks

In this paper, a novel particle swarm optimization model for radial basis function neural networks (RBFNN) using hybrid algorithms to solve classification problems is proposed. In the model, linearly decreased inertia weight of each particle (ALPSO) can be automatically calculated according to fitness value. The proposed ALPSO algorithm was compared with various well-known PSO algorithms on benchmark test functions with and without rotation. Besides, a modified fisher ratio class separability measure (MFRCSM) was used to select the initial hidden centers of radial basis function neural networks, and then orthogonal least square algorithm (OLSA) combined with the proposed ALPSO was employed to further optimize the structure of the RBFNN including the weights and controlling parameters. The proposed optimization model integrating MFRCSM, OLSA and ALPSO (MOA-RBFNN) is validated by testing various benchmark classification problems. The experimental results show that the proposed optimization method outperforms the conventional methods and approaches proposed in recent literature.

[1]  Ying Tan,et al.  Dispersed particle swarm optimization , 2008, Inf. Process. Lett..

[2]  Rudy Setiono,et al.  A note on knowledge discovery using neural networks and its application to credit card screening , 2009, Eur. J. Oper. Res..

[3]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[4]  Yang Tang,et al.  On the exponential synchronization of stochastic jumping chaotic neural networks with mixed delays and sector-bounded non-linearities , 2009, Neurocomputing.

[5]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[6]  Daniel S. Yeung,et al.  Localized Generalization Error Model and Its Application to Architecture Selection for Radial Basis Function Neural Network , 2007, IEEE Transactions on Neural Networks.

[7]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[8]  Witold Pedrycz,et al.  Conditional fuzzy clustering in the design of radial basis function neural networks , 1998, IEEE Trans. Neural Networks.

[9]  De-Shuang Huang,et al.  A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks , 2008, IEEE Transactions on Neural Networks.

[10]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[11]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[12]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[13]  Kezhi Mao,et al.  RBF neural network center selection based on Fisher ratio class separability measure , 2002, IEEE Trans. Neural Networks.

[14]  Bruce Curry,et al.  Neural networks and seasonality: Some technical considerations , 2007, Eur. J. Oper. Res..

[15]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[16]  Minqiang Li,et al.  Improving multiclass pattern recognition with a co-evolutionary RBFNN , 2008, Pattern Recognit. Lett..

[17]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[18]  Wing-Keung Wong,et al.  A decision support tool for apparel coordination through integrating the knowledge-based attribute evaluation expert system and the T-S fuzzy neural network , 2009, Expert Syst. Appl..

[19]  Wai Keung Wong,et al.  Stitching defect detection and classification using wavelet transform and BP neural network , 2009, Expert Syst. Appl..

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[22]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[24]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[25]  Wai Keung Wong,et al.  A hybrid model using genetic algorithm and neural network for classifying garment defects , 2009, Expert Syst. Appl..

[26]  Wing-Keung Wong,et al.  Fabric Stitching Inspection Using Segmented Window Technique and BP Neural Network , 2009 .

[27]  Sukhan Lee,et al.  A Gaussian potential function network with hierarchically self-organizing learning , 1991, Neural Networks.

[28]  Chandrika Kamath,et al.  Inducing oblique decision trees with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[29]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[30]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[31]  Bernhard Pfahringer,et al.  Improving on Bagging with Input Smearing , 2006, PAKDD.

[32]  Dingli Yu,et al.  Selecting radial basis function network centers with recursive orthogonal least squares training , 2000, IEEE Trans. Neural Networks Learn. Syst..