A novel self-constructing Radial Basis Function Neural-Fuzzy System

This paper proposes a novel self-constructing least-Wilcoxon generalized Radial Basis Function Neural-Fuzzy System (LW-GRBFNFS) and its applications to non-linear function approximation and chaos time sequence prediction. In general, the hidden layer parameters of the antecedent part of most traditional RBFNFS are decided in advance and the output weights of the consequent part are evaluated by least square estimation. The hidden layer structure of the RBFNFS is lack of flexibility because the structure is fixed and cannot be adjusted effectively according to the dynamic behavior of the system. Furthermore, the resultant performance of using least square estimation for output weights is often weakened by the noise and outliers. This paper creates a self-constructing scenario for generating antecedent part of RBFNFS with particle swarm optimizer (PSO). For training the consequent part of RBFNFS, instead of traditional least square (LS) estimation, least-Wilcoxon (LW) norm is employed in the proposed approach to do the estimation. As is well known in statistics, the resulting linear function by using the rank-based LW norm approximation to linear function problems is usually robust against (or insensitive to) noises and outliers and therefore increases the accuracy of the output weights of RBFNFS. Several nonlinear functions approximation and chaotic time series prediction problems are used to verify the efficiency of self-constructing LW-GRBFNIS proposed in this paper. The experimental results show that the proposed method not only creates optimal hidden nodes but also effectively mitigates the noise and outliers problems.

[1]  M. Sugeno,et al.  Structure identification of fuzzy model , 1988 .

[2]  Nanning Zheng,et al.  Self-creating and adaptive learning of RBF networks: merging soft-competition clustering algorithm with network growth technique , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[3]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[4]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[5]  Tsung-Ying Sun,et al.  Effective Learning Rate Adjustment of Blind Source Separation Based on an Improved Particle Swarm Optimizer , 2008, IEEE Transactions on Evolutionary Computation.

[6]  Meng Joo Er,et al.  Dynamic fuzzy neural networks-a novel approach to function approximation , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Yih-Lon Lin,et al.  Preliminary Study on Wilcoxon Learning Machines , 2008, IEEE Transactions on Neural Networks.

[8]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[9]  Sundaram Suresh,et al.  Meta-cognitive Neural Network for classification problems in a sequential learning framework , 2012, Neurocomputing.

[10]  Narasimhan Sundararajan,et al.  A sequential multi-category classifier using radial basis function networks , 2008, Neurocomputing.

[11]  Nicolaos B. Karayiannis,et al.  Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques , 1997, IEEE Trans. Neural Networks.

[12]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Tsung-Ying Sun,et al.  A Radial Basis Function Neural Network with Adaptive Structure via Particle Swarm Optimization , 2009 .

[14]  David Saad,et al.  Online Learning in Radial Basis Function Networks , 1997, Neural Computation.

[15]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[16]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Juan Luis Castro,et al.  Fuzzy systems with defuzzification are universal approximators , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[18]  H. J. Kim,et al.  A sequential learning algorithm for self-adaptive resource allocation network classifier , 2010, Neurocomputing.

[19]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[20]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[21]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[22]  Robert V. Hogg,et al.  Introduction to Mathematical Statistics. , 1966 .

[23]  Yunfei Bai,et al.  Genetic algorithm based self-growing training for RBF neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[24]  Tsung-Ying Sun,et al.  Cluster Distance Factor Searching by Particle Swarm Optimization for Self-Growing Radial Basis Function Neural Network , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[25]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[26]  Chein-I Chang,et al.  Robust radial basis function neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[27]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[28]  Chuen-Tsai Sun,et al.  Functional equivalence between radial basis function networks and fuzzy inference systems , 1993, IEEE Trans. Neural Networks.

[29]  Yan Li,et al.  Analysis of minimal radial basis function network algorithm for real-time identification of nonlinear dynamic systems , 2000 .

[30]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[31]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[32]  Jin-Tsong Jeng,et al.  Annealing robust radial basis function networks for function approximation with outliers , 2004, Neurocomputing.

[33]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.

[34]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[35]  David Saad,et al.  Learning and Generalization in Radial Basis Function Networks , 1995, Neural Computation.

[36]  Cheng-Han Tsai,et al.  Nonlinear Function Approximation Based on Least Wilcoxon Takagi-Sugeno Fuzzy Model , 2008, 2008 Eighth International Conference on Intelligent Systems Design and Applications.

[37]  L. Wang,et al.  Fuzzy systems are universal approximators , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[38]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.