Incremental learning of dynamic fuzzy neural networks for accurate system modeling

In this paper we propose a novel incremental learning approach based on a hybrid fuzzy neural net framework. A key feature of the approach is the adaptation of the fuzzy neural network (FNN) modeling to every new data. The typical algorithm of FNN is inefficient when used in an accurate online time series because they must be retrained from scratch every time the training set is modified. In order to reduce the expense of FNN learning for a dynamic system, a general methodology leading to quick algorithms for FNN modeling is developed. The FNN-LM algorithm for a static FNN and incremental learning algorithm (ILA) for dynamic fuzzy neural network (DFNN) are also presented to enforce the model to approximate every new sample. The ILA approach has the advantages of avoiding increasing the ranks of matrixes and avoiding solving the inverse matrix when samples increase gradually. When it is used to predict an accurate online time series, the DFNN model can efficiently update a trained static FNN with a very fast speed according to the sample added to the training set. Numerical experiments validate our theoretical results. Excellent performances of the proposed approach in modeling accuracy and learning convergence are exhibited.

[1]  Puyin Liu,et al.  Universal approximations of continuous fuzzy-valued functions by multi-layer regular fuzzy neural networks , 2001, Fuzzy Sets Syst..

[2]  Ignacio Rojas,et al.  Short-Term Prediction of Chaotic Time Series by Using RBF Network with Regression Weights , 2000, Int. J. Neural Syst..

[3]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[4]  Xinghuo Yu,et al.  A fuzzy neural network approximator with fast terminal sliding mode and its applications , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[5]  Kwang Bo Cho,et al.  Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction , 1996, Fuzzy Sets Syst..

[6]  Chunshien Li,et al.  Recurrent neuro-fuzzy hybrid-learning approach to accurate system modeling , 2007, Fuzzy Sets Syst..

[7]  M. MendelJ.,et al.  Fuzzy basis functions , 1995 .

[8]  Meng Joo Er,et al.  NARMAX time series model prediction: feedforward and recurrent fuzzy neural network approaches , 2005, Fuzzy Sets Syst..

[9]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[10]  Mario Martín Muñoz On-line support vector machines for function approximation , 2002 .

[11]  Yong-Zai Lu,et al.  Fuzzy Model Identification and Self-Learning for Dynamic Systems , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  E. H. Mamdani,et al.  Application of Fuzzy Logic to Approximate Reasoning Using Linguistic Synthesis , 1976, IEEE Transactions on Computers.

[13]  Kazuo Tanaka,et al.  Successive identification of a fuzzy model and its applications to prediction of a complex system , 1991 .

[14]  Ralf Östermark,et al.  A hybrid genetic fuzzy neural network algorithm designed for classification problems involving several groups , 2000, Fuzzy Sets Syst..

[15]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[16]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[17]  Chia-Feng Juang,et al.  Temporal problems solved by dynamic fuzzy network based on genetic algorithm with variable-length chromosomes , 2004, Fuzzy Sets Syst..

[18]  Da Ruan,et al.  Novel neural algorithms based on fuzzy δ rules for solving fuzzy relation equations: Part I , 1997, Fuzzy Sets Syst..

[19]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[20]  Paris A. Mastorocostas,et al.  FUNCOM: A constrained learning algorithm for fuzzy neural networks , 2000, Fuzzy Sets Syst..

[21]  Nikola K. Kasabov,et al.  HyFIS: adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems , 1999, Neural Networks.

[22]  James Theiler,et al.  Accurate On-line Support Vector Regression , 2003, Neural Computation.

[23]  T. Martin McGinnity,et al.  An approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network , 2005, Fuzzy Sets Syst..

[24]  Ralf Östermark,et al.  A fuzzy neural network algorithm for multigroup classification , 1999, Fuzzy Sets Syst..

[25]  Meng Joo Er,et al.  A fast learning algorithm for parsimonious fuzzy neural systems , 1999, 1999 European Control Conference (ECC).

[26]  Ruowei Zhou,et al.  Structure and learning algorithms of a nonsingleton input fuzzy neural network based on the approximate analogical reasoning schema , 2006, Fuzzy Sets Syst..

[27]  Chin-Teng Lin,et al.  A neural fuzzy network for word information processing , 2002, Fuzzy Sets Syst..

[28]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[29]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.