Optimizing widths with PSO for center selection of Gaussian radial basis function networks

The radial basis function (RBF) centers play different roles in determining the classification capability of a Gaussian radial basis function neural network (GRBFNN) and should hold different width values. However, it is very hard and time-consuming to optimize the centers and widths at the same time. In this paper, we introduce a new insight into this problem. We explore the impact of the definition of widths on the selection of the centers, propose an optimization algorithm of the RBF widths in order to select proper centers from the center candidate pool, and improve the classification performance of the GRBFNN. The design of the objective function of the optimization algorithm is based on the local mapping capability of each Gaussian RBF. Further, in the design of the objective function, we also handle the imbalanced problem which may occur even when different local regions have the same number of examples. Finally, the recursive orthogonal least square (ROLS) and genetic algorithm (GA), which are usually adopted to optimize the RBF centers, are separately used to select the centers from the center candidates with the initialized widths, in order to testify the validity of our proposed width initialization strategy on the selection of centers. Our experimental results show that, compared with the heuristic width setting method, the width optimization strategy makes the selected centers more appropriate, and improves the classification performance of the GRBFNN. Moreover, the GRBFNN constructed by our method can attain better classification performance than the RBF LS-SVM, which is a state-of-the-art classifier.

[1]  Lipo Wang,et al.  Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Sheng Chen,et al.  Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks , 1999, IEEE Trans. Neural Networks.

[3]  Imran Sarwar Bajwa,et al.  Feature Based Image Classification by using Principal Component Analysis , 2009 .

[4]  De-Shuang Huang,et al.  A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks , 2008, IEEE Transactions on Neural Networks.

[5]  Mário A. T. Figueiredo On Gaussian radial basis function approximations: interpretation, extensions, and learning strategies , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[6]  Zhi-Hua Zhou,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Training Cost-sensitive Neural Networks with Methods Addressing the Class Imbalance Problem , 2022 .

[7]  De-Shuang Huang,et al.  Human face recognition based on multi-features using neural networks committee , 2004, Pattern Recognit. Lett..

[8]  P. Deb Finite Mixture Models , 2008 .

[9]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[10]  Yizhen Huang,et al.  Learning from interpolated images using neural networks for digital forensics , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Gianni Costa,et al.  From global to local and viceversa: uses of associative rule learning for classification in imprecise environments , 2011, Knowledge and Information Systems.

[12]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[13]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[14]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[15]  Daibing Zhang,et al.  Design of an artificial bionic neural network to control fish-robot's locomotion , 2008, Neurocomputing.

[16]  Zhong-Qiu Zhao,et al.  A novel modular neural network for imbalanced classification problems , 2009, Pattern Recognit. Lett..

[17]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary optimization of RBF networks , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[18]  Daniel S. Yeung,et al.  Localized Generalization Error Model and Its Application to Architecture Selection for Radial Basis Function Neural Network , 2007, IEEE Transactions on Neural Networks.

[19]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[20]  R. Gupta,et al.  Maximum power point tracking of multiple photovoltaic arrays , 2012, 2012 Students Conference on Engineering and Systems.

[21]  Zulkifli Mohd Nopiah,et al.  Time complexity estimation and optimisation of the genetic algorithm clustering method , 2010 .

[22]  Tae-Sun Choi,et al.  Impulse noise filtering based on noise-free pixels using genetic programming , 2011, Knowledge and Information Systems.

[23]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[24]  Dingli Yu,et al.  Selecting radial basis function network centers with recursive orthogonal least squares training , 2000, IEEE Trans. Neural Networks Learn. Syst..

[25]  Meng Joo Er,et al.  Face recognition with radial basis function (RBF) neural networks , 2002, IEEE Trans. Neural Networks.

[26]  Shitong Wang,et al.  Advanced fuzzy cellular neural network: Application to CT liver images , 2007, Artif. Intell. Medicine.

[27]  Djamel Bouchaffra,et al.  Genetic-based EM algorithm for learning Gaussian mixture models , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[29]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[30]  José Martínez Sotoca,et al.  An Empirical Study for the Multi-class Imbalance Problem with Neural Networks , 2008, CIARP.

[31]  Xia Hong,et al.  A New RBF Neural Network With Boundary Value Constraints , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[32]  Jorge Bondia,et al.  New possibilistic method for discovering linear local behavior using hyper-Gaussian distributed membership function , 2011, Knowledge and Information Systems.

[33]  Cuneyt Guzelis,et al.  Input-output clustering for determining centers of radial basis function network , 1997 .

[34]  Masafumi Miyatake,et al.  Maximum Power Point Tracking of Multiple Photovoltaic Arrays: A PSO Approach , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[35]  Hao Wang,et al.  An new immune genetic algorithm based on uniform design sampling , 2012, Knowledge and Information Systems.