Evolving Fuzzy Systems from Data Streams in Real-Time

An approach to real-time generation of fuzzy rule-base systems of extended Takagi-Sugeno (xTS) type from data streams is proposed in the paper. The xTS fuzzy system combines both zero and first order Takagi-Sugeno (TS) type systems. The fuzzy rule-base (system structure) evolves starting 'from scratch' based on the data distribution in the joint input/output data space. An incremental clustering procedure that takes into account the non-stationary nature of the data pattern and generates clusters that are used to form fuzzy rule based systems antecedent part in on-line mode is used as a first stage of the non-iterative learning process. This structure proved to be computationally efficient and powerful to represent in a transparent way complex non-linear relationships. The decoupling of the learning task into a non-iterative, recursive (thus computationally very efficient and applicable in real-time) clustering with a modified version of the well known recursive parameter estimation technique leads to a very powerful construct - evolving xTS (exTS). It is transparent and linguistically interpretable. The contributions of this paper are: i) introduction of an adaptive recursively updated radius of the clusters (zone of influence of the fuzzy rules) that learns the data distribution/variance/scatter in each cluster; ii) a new condition to replace clusters that excludes contradictory rules; iii) an extended formulation that includes both zero order TS and simplified Mamdani multi-input-multi-output (MIMO) systems; iv) new improved formulation of the membership functions, which closer resembles the normal Gaussian distribution; v) introduction of measures of clusters quality that are used to form the antecedent parts of respective fuzzy rules, namely their age and support; vi) experimental results with a well known benchmark problem as well as with real experimental data of concentration of exhaust gases (NOx) in on-line modeling of car engine test rigs

[1]  Ebrahim H. Mamdani,et al.  A linguistic self-organizing process controller , 1979, Autom..

[2]  Karl Johan Åström,et al.  Computer-Controlled Systems: Theory and Design , 1984 .

[3]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[5]  PlattJohn.  A resource-allocating network for function interpolation , 1991 .

[6]  Ronald R. Yager,et al.  Learning of Fuzzy Rules by Mountain Clustering , 1992 .

[7]  L. Wang,et al.  Fuzzy systems are universal approximators , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[8]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[9]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[10]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[11]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[12]  T. Fukuda,et al.  Self-tuning fuzzy modeling with adaptive membership function, rules, and hierarchical structure based on genetic algorithm , 1995 .

[13]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[14]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[15]  Chia-Feng Juang,et al.  A recurrent self-organizing neural fuzzy inference network , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[16]  Chin-Teng Lin,et al.  A recurrent self-organizing neural fuzzy inference network , 1999, IEEE Trans. Neural Networks.

[17]  Peter J. Bentley.  Evolving Fuzzy Detectives: An Investigation into the Evolution of Fuzzy Rules , 1999 .

[18]  Dimitar Petrov Filev.  Rule-base guided adaptation for mode detection in process control , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[19]  Nikola K. Kasabov,et al.  Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[20]  Chih-Hong Lin,et al.  Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive , 2001, IEEE Trans. Fuzzy Syst..

[21]  P. Angelov,et al.  Evolving rule-based models: A tool for intelligent adaptation , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[22]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[23]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Dimitar Filev,et al.  On-Line Evolution of Takagi-Sugeno Fuzzy Models , 2004, IFAC Proceedings Volumes.

[25]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[26]  Plamen P. Angelov,et al.  Simpl_eTS: a simplified method for learning evolving Takagi-Sugeno fuzzy models , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[27]  P. Angelov,et al.  Two approaches to data-driven design of evolving fuzzy systems: eTS and FLEXFIS , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[28]  T. Martin McGinnity,et al.  An approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network , 2005, Fuzzy Sets Syst..

[29]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.