Improving the effect of fuzzy clustering on RBF network's performance in terms of particle swarm optimization

This paper quantifies the effect of fuzzy clustering in the design process of a typical RBF network.It is analytically shown that the fuzzy clustering acts to minimize an upper bound of the network's square error.The PSO algorithm is used to minimize the upper bound and to provide an estimation of the network's parameters.Finally, the widths and connection weights are further tuned using a steepest descent approach. This paper proposes a novel training algorithm for radial basis function neural networks based on fuzzy clustering and particle swarm optimization. So far, fuzzy clustering has proven to be a very efficient tool in designing such kind of networks. The motivation of the current work is to quantify the exact effect of fuzzy cluster analysis on the network's performance and use it in order to substantially improve this performance. There are two key theoretical findings resulting from the present work. First, it is analytically proved that when the standard fuzzy c-means algorithm is used to generate the input space fuzzy partition, the main effect this partition imposes to the network's square error (i.e. performance index) can be written down in terms of a distortion function that measures the ability of the partition to recreate the original data. Second, using the aforementioned distortion function, an upper bound of the network's square error can be constructed. Then, the particle swarm optimization (PSO) is put in place to minimize the above upper bound and determine the network's parameters. To further improve the accuracy, the basis function widths and the connection weights are fine-tuned by employing a steepest descent approach. The main experimental findings are: (a) the implementation of the PSO obtains a significant reduction of the square error while exhibiting a smooth dynamic behavior, (b) although the steepest descent further decreases the error it finally obtains smaller reduction rates, meaning that the strongest impact on the error reduction is provided by the PSO, and (c) the improved performance of the proposed network is demonstrated through an extensive comparison with other related methods using a 10-fold cross-validation analysis.

[1]  Wai Keung Wong,et al.  A hybrid particle swarm optimization and its application in neural networks , 2012, Expert Syst. Appl..

[2]  Witold Pedrycz,et al.  Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering , 2006, Neurocomputing.

[3]  Okyay Kaynak,et al.  Robust and adaptive backstepping control for nonlinear systems using RBF neural networks , 2004, IEEE Transactions on Neural Networks.

[4]  Chia-Nan Ko,et al.  Time series prediction using RBF neural networks with a nonlinear time-varying evolution PSO algorithm , 2009, Neurocomputing.

[5]  Zekeriya Uykan,et al.  Analysis of input-output clustering for determining centers of RBFN , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  Witold Pedrycz,et al.  Linguistic models as a framework of user-centric system modeling , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[7]  Chris T. Kiranoudis,et al.  Classification of aged wine distillates using fuzzy and neural network systems , 2000 .

[8]  Efstratios F. Georgopoulos,et al.  Forecasting foreign exchange rates with adaptive neural networks using radial-basis functions and Particle Swarm Optimization , 2013, Eur. J. Oper. Res..

[9]  Chin-Wang Tao,et al.  Radial Basis Function Networks With Linear Interval Regression Weights for Symbolic Interval Data , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Siti Zaiton Mohd Hashim,et al.  Memetic multiobjective particle swarm optimization-based radial basis function network for classification problems , 2013, Inf. Sci..

[11]  Witold Pedrycz,et al.  Fuzzy vector quantization with the particle swarm optimization: A study in fuzzy granulation-degranulation information processing , 2007, Signal Process..

[12]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[13]  Cheng-Han Tsai,et al.  A novel self-constructing Radial Basis Function Neural-Fuzzy System , 2013, Appl. Soft Comput..

[14]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[15]  Xiao-Jun Zeng,et al.  A clustering algorithm for radial basis function neural network initialization , 2012, Neurocomputing.

[16]  George E. Tsekouras,et al.  A novel training algorithm for RBF neural network using a hybrid fuzzy clustering approach , 2012, Fuzzy Sets Syst..

[17]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[18]  Sung-Kwun Oh,et al.  A comparative experimental study of type-1/type-2 fuzzy cascade controller based on genetic algorithms and particle swarm optimization , 2011, Expert Syst. Appl..

[19]  Karim Faez,et al.  A fuzzy hybrid learning algorithm for radial basis function neural network with application in human face recognition , 2003, Pattern Recognit..

[20]  Zheng Rong Yang,et al.  A novel radial basis function neural network for discriminant analysis , 2006, IEEE Transactions on Neural Networks.

[21]  Changhe Li,et al.  A Self-Learning Particle Swarm Optimizer for Global Optimization Problems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Yan-Ping Zhou,et al.  Adaptive configuration of radial basis function network by regression tree allied with hybrid particle swarm optimization algorithm , 2013 .

[23]  Dilip Kumar Pratihar,et al.  Tuning of neural networks using particle swarm optimization to model MIG welding process , 2011, Swarm Evol. Comput..

[24]  John Tsimikas,et al.  On training RBF neural networks using input-output fuzzy clustering and particle swarm optimization , 2013, Fuzzy Sets Syst..

[25]  Mustafa A. Akcoglu,et al.  A course in advanced calculus , 2009 .

[26]  Gholam Ali Montazer,et al.  An improvement in RBF learning algorithm based on PSO for real time applications , 2013, Neurocomputing.

[27]  Witold Pedrycz,et al.  Conditional fuzzy clustering in the design of radial basis function neural networks , 1998, IEEE Trans. Neural Networks.

[28]  Sung-Kwun Oh,et al.  Granular Neural Networks and Their Development Through Context-Based Clustering and Adjustable Dimensionality of Receptive Fields , 2009, IEEE Transactions on Neural Networks.

[29]  Hsuan-Ming Feng,et al.  Self-generation RBFNs using evolutional PSO learning , 2006, Neurocomputing.