Evolving fuzzy optimally pruned extreme learning machine for regression problems

This paper proposes an approach to the identification of evolving fuzzy Takagi–Sugeno systems based on the optimally pruned extreme learning machine (OP-ELM) methodology. First, we describe ELM, a simple yet accurate learning algorithm for training single-hidden layer feed-forward artificial neural networks with random hidden neurons. We then describe the OP-ELM methodology for building ELM models in a robust and simplified manner suitable for evolving approaches. Based on the previously proposed ELM method, and the OP-ELM methodology, we propose an identification method for self-developing or evolving neuro-fuzzy systems applicable to regression problems. This method, evolving fuzzy optimally pruned extreme learning machine (eF-OP-ELM), follows a random projection based approach to extracting evolving fuzzy rulebases. In this approach systems are not only evolving but their structure is defined on the basis of randomly generated fuzzy basis functions. A comparative analysis of eF-OP-ELM is performed over a diverse collection of benchmark datasets against well known evolving neuro-fuzzy methods, namely eTS and DENFIS. Results show that the method proposed yields compact rulebases, is robust and competitive in terms of accuracy.

[1]  A. Dourado,et al.  Pruning for interpretability of large spanned eTS , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[2]  Amaury Lendasse,et al.  Fuzzy inference based autoregressors for time series prediction using nonparametric residual variance estimation , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[3]  Paramasivan Saratchandran,et al.  Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction , 2006, Fuzzy Sets Syst..

[4]  A. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[5]  Fernando Gomide,et al.  Interval-based evolving modeling , 2009, 2009 IEEE Workshop on Evolving and Self-Developing Intelligent Systems.

[6]  Dimitris Achlioptas,et al.  Database-friendly random projections , 2001, PODS.

[7]  Timo Similä,et al.  Multiresponse Sparse Regression with Application to Multidimensional Scaling , 2005, ICANN.

[8]  Guang-Bin Huang,et al.  Reply to “Comments on “The Extreme Learning Machine”” , 2008, IEEE Transactions on Neural Networks.

[9]  Sergey Kravtsov,et al.  A new dynamical mechanism for major climate shifts , 2007 .

[10]  Iluminada Baturone,et al.  Automatic tuning of complex fuzzy systems with Xfuzzy , 2007, Fuzzy Sets Syst..

[11]  Dmitriy Fradkin,et al.  Experiments with random projections for machine learning , 2003, KDD '03.

[12]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Amaury Lendasse,et al.  OP-ELM: Theory, Experiments and a Toolbox , 2008, ICANN.

[14]  Amaury Lendasse,et al.  Long-term prediction of time series using NNE-based projection and OP-ELM , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[15]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[16]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[17]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[18]  Lotfi A. Zadeh,et al.  Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic , 1997, Fuzzy Sets Syst..

[19]  Xiao-Jun Zeng,et al.  Approximation theory of fuzzy systems-MIMO case , 1995, IEEE Trans. Fuzzy Syst..

[20]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[21]  John C. Platt A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[22]  Xiao-Jun Zeng,et al.  Approximation theory of fuzzy systems-SISO case , 1994, IEEE Trans. Fuzzy Syst..

[23]  Plamen P. Angelov,et al.  Simpl_eTS: a simplified method for learning evolving Takagi-Sugeno fuzzy models , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[24]  Plamen P. Angelov,et al.  Guest Editorial Evolving Fuzzy Systems–-Preface to the Special Section , 2008, IEEE Transactions on Fuzzy Systems.

[25]  J. Victor Ramos,et al.  eFSLab: Developing evolving fuzzy systems from data in a friendly environment , 2009, 2009 European Control Conference (ECC).

[26]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[27]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[28]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[29]  N. Kasabov,et al.  Evolving Systems , 2005 .

[30]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[31]  Amaury Lendasse,et al.  Autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation , 2010, Fuzzy Sets Syst..

[32]  Zhaohua Wu,et al.  On the trend, detrending, and variability of nonlinear and nonstationary time series , 2007, Proceedings of the National Academy of Sciences.

[33]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[34]  FengGuorui,et al.  Error minimized extreme learning machine with growth of hidden nodes and incremental learning , 2009 .

[35]  Amaury Lendasse,et al.  Machine Learning Techniques based on Random Projections , 2010, ESANN.

[36]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[37]  R. H. Myers Classical and modern regression with applications , 1986 .

[38]  Mauro Birattari,et al.  Lazy Learning Meets the Recursive Least Squares Algorithm , 1998, NIPS.

[39]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[40]  F. Montesino,et al.  xftsp: A tool for time series prediction by means of fuzzy inference systems , 2008, 2008 4th International IEEE Conference Intelligent Systems.

[41]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[42]  Plamen Angelov,et al.  Evolving Intelligent Systems: Methodology and Applications , 2010 .

[43]  Plamen Angelov,et al.  Evolving Intelligent Systems, eIS , 2006 .

[44]  James W. Hurrell,et al.  North Atlantic climate variability: The role of the North Atlantic Oscillation , 2009 .

[45]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[46]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[47]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[48]  Edwin Lughofer,et al.  FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi–Sugeno Fuzzy Models , 2008, IEEE Transactions on Fuzzy Systems.

[49]  Plamen P. Angelov,et al.  Flexible models with evolving structure , 2004, Int. J. Intell. Syst..

[50]  John Q. Gan,et al.  Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling , 2008, Fuzzy Sets Syst..

[51]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[52]  Witold Pedrycz,et al.  Knowledge-based clustering - from data to information granules , 2007 .

[53]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[54]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[55]  Ronald R. Yager,et al.  Measures of specificity over continuous spaces under similarity relations , 2008, Fuzzy Sets Syst..

[56]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .