Generalized Hybrid Constructive Learning Algorithm for Multioutput RBF Networks

An efficient generalized hybrid constructive (GHC) learning algorithm for multioutput radial basis function (RBF) networks is proposed to obtain a compact network with good generalization capability. By this algorithm, one can train the adjustable parameters and determine the optimal network structure simultaneously. First, an initialization method based on the growing and pruning algorithm is utilized to select the important initial hidden neurons and candidate ones. Then, by introducing a generalized hidden matrix, a structured parameter optimization algorithm is presented to train multioutput RBF network with fixed size, which combines Levenberg-Marquardt (LM) algorithm with least-square method together. Beginning from an appropriate number of hidden neurons, new neurons chosen from the candidates are added one by one each time when the training entraps into local minima. By incorporating an improved incremental constructive scheme, the training is built on previous results after adding new neurons such that the GHC learning algorithm avoids a trial-and-error procedure. Furthermore, based on the improved computation for LM training, the memory limitation problem is solved. The computational complexity analysis and experimental results demonstrate that better performance is efficiently achieved by this algorithm.

[1]  De-Shuang Huang,et al.  Determining the centers of radial basis probabilistic neural networks by recursive orthogonal least square algorithms , 2005, Appl. Math. Comput..

[2]  Nicolaos B. Karayiannis,et al.  On the construction and training of reformulated radial basis function neural networks , 2003, IEEE Trans. Neural Networks.

[3]  Yu Gong,et al.  Online Modeling With Tunable RBF Network , 2013, IEEE Transactions on Cybernetics.

[4]  E. Süli,et al.  An introduction to numerical analysis , 2003 .

[5]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[6]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[7]  George W. Irwin,et al.  A hybrid linear/nonlinear training algorithm for feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[8]  H. Akaike.  A new look at the statistical model identification , 1974 .

[9]  Manel Guerrero Zapata,et al.  A hybrid multiobjective RBF-PSO method for mitigating DoS attacks in Named Data Networking , 2015, Neurocomputing.

[10]  Yaohua Xiong,et al.  Training Reformulated Radial Basis Function Neural Networks Capable of Identifying Uncertainty in Data Classification , 2006, IEEE Transactions on Neural Networks.

[11]  Yu Gong,et al.  A Fast Adaptive Tunable RBF Network For Nonstationary Systems , 2016, IEEE Transactions on Cybernetics.

[12]  Hao Yu,et al.  Improved Computation for Levenberg–Marquardt Training , 2010, IEEE Transactions on Neural Networks.

[13]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[14]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[15]  Lipo Wang,et al.  Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[16]  Hamid Reza Karimi,et al.  Adaptive Neural Stabilizing Controller for a Class of Mismatched Uncertain Nonlinear Systems by State and Output Feedback , 2015, IEEE Transactions on Cybernetics.

[17]  Habib Rajabi Mashhadi,et al.  A fast static security assessment method based on radial basis function neural networks using enhanced clustering , 2013 .

[18]  Sheng Chen,et al.  Nonlinear Identification Using Orthogonal Forward Regression With Nested Optimal Regularization , 2015, IEEE Transactions on Cybernetics.

[19]  Dan Simon,et al.  Training radial basis neural networks with the extended Kalman filter , 2002, Neurocomputing.

[20]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[21]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[22]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[23]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[24]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[25]  Gholam Ali Montazer,et al.  Improvement of RBF neural networks using Fuzzy-OSD algorithm in an online radar pulse classification system , 2013, Appl. Soft Comput..

[26]  Yukihiro Toyoda,et al.  A parameter optimization method for radial basis function type models , 2003, IEEE Trans. Neural Networks.

[27]  Paramasivan Saratchandran,et al.  Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm , 1998, IEEE Trans. Neural Networks.

[28]  Dimitris Gavrilis,et al.  Real-time detection of distributed denial-of-service attacks using RBF networks and statistical features , 2005, Comput. Networks.

[29]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.

[30]  Shui-Chun Lin,et al.  Adaptive Neural Network Control of a Self-balancing Two-wheeled Scooter , 2007, IECON 2007 - 33rd Annual Conference of the IEEE Industrial Electronics Society.

[31]  Patrick van der Smagt.  Minimisation methods for training feedforward neural networks , 1994, Neural Networks.

[32]  George W. Irwin,et al.  A Novel Continuous Forward Algorithm for RBF Neural Modelling , 2007, IEEE Transactions on Automatic Control.

[33]  Bogdan M. Wilamowski,et al.  A Hybrid Constructive Algorithm for Single-Layer Feedforward Networks Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[34]  George W. Irwin,et al.  A New Jacobian Matrix for Optimal Learning of Single-Layer Neural Networks , 2008, IEEE Transactions on Neural Networks.

[35]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[36]  Bing Lam Luk,et al.  Construction of Tunable Radial Basis Function Networks Using Orthogonal Forward Selection , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[37]  Min Gan,et al.  A Variable Projection Approach for Efficient Estimation of RBF-ARX Model , 2015, IEEE Transactions on Cybernetics.

[38]  H. B. Nielsen.  DAMPING PARAMETER IN MARQUARDT ’ S METHOD , 1999 .

[39]  Shuzhi Sam Ge,et al.  Adaptive neural control of uncertain MIMO nonlinear systems , 2004, IEEE Transactions on Neural Networks.

[40]  Yen-Jen Oyang,et al.  Data classification with radial basis function networks based on a novel kernel density estimation algorithm , 2005, IEEE Transactions on Neural Networks.