A Structure-Adaptive Hybrid RBF-BP Classifier with an Optimized Learning Strategy

This paper presents a structure-adaptive hybrid RBF-BP (SAHRBF-BP) classifier with an optimized learning strategy. SAHRBF-BP is composed of a structure-adaptive RBF network and a BP network of cascade, where the number of RBF hidden nodes is adjusted adaptively according to the distribution of sample space, the adaptive RBF network is used for nonlinear kernel mapping and the BP network is used for nonlinear classification. The optimized learning strategy is as follows: firstly, a potential function is introduced into training sample space to adaptively determine the number of initial RBF hidden nodes and node parameters, and a form of heterogeneous samples repulsive force is designed to further optimize each generated RBF hidden node parameters, the optimized structure-adaptive RBF network is used for adaptively nonlinear mapping the sample space; then, according to the number of adaptively generated RBF hidden nodes, the number of subsequent BP input nodes can be determined, and the overall SAHRBF-BP classifier is built up; finally, different training sample sets are used to train the BP network parameters in SAHRBF-BP. Compared with other algorithms applied to different data sets, experiments show the superiority of SAHRBF-BP. Especially on most low dimensional and large number of data sets, the classification performance of SAHRBF-BP outperforms other training SLFNs algorithms.

[1]  Joni-Kristian Kämäräinen,et al.  Differential Evolution Training Algorithm for Feed-Forward Neural Networks , 2003, Neural Processing Letters.

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Visakan Kadirkamanathan,et al.  A Function Estimation Approach to Sequential Learning with Neural Networks , 1993, Neural Computation.

[4]  Julio Ortega Lopera,et al.  Improved RAN sequential prediction using orthogonal techniques , 2001, Neurocomputing.

[5]  Zhiping Lin,et al.  Self-Adaptive Evolutionary Extreme Learning Machine , 2012, Neural Processing Letters.

[6]  Seok-Beom Roh,et al.  The design methodology of radial basis function neural networks based on fuzzy K-nearest neighbors approach , 2010, Fuzzy Sets Syst..

[7]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[8]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[9]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[10]  Witold Pedrycz,et al.  Conditional fuzzy clustering in the design of radial basis function neural networks , 1998, IEEE Trans. Neural Networks.

[11]  Adrian G. Bors,et al.  Minimal Topology for a Radial Basis Functions Neural Network for Pattern Classification , 1994 .

[12]  George Panoutsos,et al.  Interval Type-2 Radial Basis Function Neural Network: A Modeling Framework , 2015, IEEE Transactions on Fuzzy Systems.

[13]  Narasimhan Sundararajan,et al.  A Metacognitive Neuro-Fuzzy Inference System (McFIS) for Sequential Classification Problems , 2013, IEEE Transactions on Fuzzy Systems.

[14]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[15]  W. S. Meisel,et al.  Potential Functions in Mathematical Pattern Recognition , 1969, IEEE Transactions on Computers.

[16]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[17]  Nicolaos B. Karayiannis,et al.  Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques , 1997, IEEE Trans. Neural Networks.

[18]  S. Fatikow,et al.  Bi-directional evolutionary level set method for topology optimization , 2015 .

[19]  Aristidis Likas,et al.  An incremental training method for the probabilistic RBF network , 2006, IEEE Trans. Neural Networks.

[20]  H. J. Kim,et al.  A sequential learning algorithm for self-adaptive resource allocation network classifier , 2010, Neurocomputing.

[21]  Witold Pedrycz,et al.  Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering , 2006, Neurocomputing.

[22]  Sundaram Suresh,et al.  Metacognitive Learning in a Fully Complex-Valued Radial Basis Function Neural Network , 2012, Neural Computation.

[23]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[24]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[25]  W. Zhuang,et al.  Integrating Local and Global Error Statistics for Multi-Scale RBF Network Training: An Assessment on Remote Sensing Data , 2012, PloS one.

[26]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[27]  Hao Yu,et al.  Fast and Efficient Second-Order Method for Training Radial Basis Function Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[28]  David Lowe,et al.  Characterising complexity by the degrees of freedom in a radial basis function network , 1998, Neurocomputing.

[29]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[30]  Hao Yu,et al.  An Incremental Design of Radial Basis Function Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Marimuthu Palaniswami,et al.  Effects of moving the center's in an RBF network , 2002, IEEE Trans. Neural Networks.

[32]  Maria Bortman,et al.  A Growing and Pruning Method for Radial Basis Function Networks , 2009, IEEE Transactions on Neural Networks.

[33]  Sung-Kwun Oh,et al.  Design of K-means clustering-based polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution , 2012, Neurocomputing.

[34]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[35]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[36]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[37]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[38]  Bidyadhar Subudhi,et al.  Differential Evolution and Levenberg Marquardt Trained Neural Network Scheme for Nonlinear System Identification , 2008, Neural Processing Letters.

[39]  Marimuthu Palaniswami,et al.  Effects of moving the centers in an RBF network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).