Fuzzy evolving linear regression trees

This paper introduces a new approach for evolving fuzzy modeling using tree structures. The model is a fuzzy linear regression tree whose topology can be continuously updated through a statistical model selection test. A fuzzy linear regression tree is a fuzzy tree with linear model in each leaf. An incremental learning algorithm approach evolves the tree replacing leaves with subtrees that improve the model quality. The learning algorithm evaluates each input only once and do not need to store any past values. The evolving linear regression model is evaluated using time series forecasting problems. The performance is compared against alternative evolving fuzzy models and classic models with fixed structures. The results suggest that fuzzy evolving regression tree is a promising approach for adaptive system modeling.

[1]  Nikola Kasabov,et al.  Evolving Intelligent Systems: Methods, Learning, & Applications , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[2]  L. Rabiner,et al.  The acoustics, speech, and signal processing society - A historical perspective , 1984, IEEE ASSP Magazine.

[3]  Cezary Z. Janikow,et al.  Fuzzy decision trees: issues and methods , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[4]  João Gama,et al.  Learning Model Trees from Data Streams , 2008, Discovery Science.

[5]  José de Jesús Rubio,et al.  SOFMLS: Online Self-Organizing Fuzzy Modified Least-Squares Network , 2009, IEEE Transactions on Fuzzy Systems.

[6]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[7]  Xiao-Jun Zeng,et al.  A structure evolving learning method for fuzzy systems , 2010, Evol. Syst..

[8]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[9]  Peter C. Young,et al.  Recursive Estimation and Time-Series Analysis: An Introduction , 1984 .

[10]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[11]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[12]  Plamen Angelov,et al.  Algorithms for Real-Time Clustering and Generation of Rules from Data , 2007 .

[13]  João Gama,et al.  Regression Trees from Data Streams with Drift Detection , 2009, Discovery Science.

[14]  Duncan Potts,et al.  Incremental learning of linear model trees , 2004, ICML.

[15]  Witold Pedrycz,et al.  Fuzzy Systems Engineering , 2007 .

[16]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[17]  Witold Pedrycz,et al.  Fuzzy Systems Engineering - Toward Human-Centric Computing , 2007 .

[18]  Witold Pedrycz,et al.  Advances in Fuzzy Clustering and its Applications , 2007 .

[19]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[20]  Nikola Kasabov,et al.  Foundations Of Neural Networks, Fuzzy Systems, And Knowledge Engineering [Books in Brief] , 1996, IEEE Transactions on Neural Networks.

[21]  Jerry M. Mendel,et al.  Generating fuzzy rules by learning from examples , 1992, IEEE Trans. Syst. Man Cybern..

[22]  Claude Sammut,et al.  Incremental Learning of Linear Model Trees , 2004, Machine Learning.

[23]  P. Angelov,et al.  Evolving Fuzzy Systems from Data Streams in Real-Time , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[24]  Chin-Teng Lin,et al.  An online self-constructing neural fuzzy inference network and its applications , 1998, IEEE Trans. Fuzzy Syst..

[25]  Johannes Gehrke,et al.  SECRET: a scalable linear regression tree algorithm , 2002, KDD.

[26]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[27]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[28]  Edwin Lughofer,et al.  FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi–Sugeno Fuzzy Models , 2008, IEEE Transactions on Fuzzy Systems.

[29]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[30]  Amaury Lendasse,et al.  Evolving fuzzy optimally pruned extreme learning machine for regression problems , 2010, Evol. Syst..

[31]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[32]  Rupert G. Miller.  Simultaneous Statistical Inference , 1966 .

[33]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[34]  Douglas C. Montgomery,et al.  Introduction to Statistical Quality Control , 1986 .

[35]  Saifur Rahman,et al.  A generalized knowledge-based short-term load-forecasting technique , 1993 .

[36]  Luís Torgo,et al.  Functional Models for Regression Tree Leaves , 1997, ICML.

[37]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[38]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Plamen Angelov,et al.  A genetic-algorithm-based approach to optimization of bioprocesses described by fuzzy rules , 1997 .

[40]  David G. Stork,et al.  Pattern Classification , 1973 .

[41]  R. Yager,et al.  Approximate Clustering Via the Mountain Method , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[42]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[43]  Plamen P. Angelov,et al.  Simpl_eTS: a simplified method for learning evolving Takagi-Sugeno fuzzy models , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[44]  O. Nelles.  Nonlinear System Identification , 2001 .

[45]  Plamen P. Angelov,et al.  Guest Editorial Evolving Fuzzy Systems–-Preface to the Special Section , 2008, IEEE Transactions on Fuzzy Systems.

[46]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[47]  Robert Babuska,et al.  Fuzzy Modeling for Control , 1998 .

[48]  Edwin Lughofer,et al.  Extensions of vector quantization for incremental clustering , 2008, Pattern Recognit..