Data-Driven Fuzzy Modeling Using Deep Learning

Fuzzy modeling has many advantages over the non-fuzzy methods, such as robustness against uncertainties and less sensitivity to the varying dynamics of nonlinear systems. Data-driven fuzzy modeling needs to extract fuzzy rules from the input/output data, and train the fuzzy parameters. This paper takes advantages from deep learning, probability theory, fuzzy modeling, and extreme learning machines. We use the restricted Boltzmann machine (RBM) and probability theory to overcome some common problems in data based modeling methods. The RBM is modified such that it can be trained with continuous values. A probability based clustering method is proposed to partition the hidden features from the RBM, and extract fuzzy rules with probability measurement. An extreme learning machine and an optimization method are applied to train the consequent part of the fuzzy rules and the probability parameters. The proposed method is validated with two benchmark problems.

[1]  Spyros G. Tzafestas,et al.  NeuroFAST: on-line neuro-fuzzy ART-based structure and parameter learning TSK model , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Li-Xin Wang,et al.  Adaptive fuzzy systems and control , 1994 .

[3]  Gang Chen,et al.  Deep Learning with Nonparametric Clustering , 2015, ArXiv.

[4]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[5]  Xiaoou Li,et al.  Nonlinear system identification using deep learning and randomized algorithms , 2015, 2015 IEEE International Conference on Information and Automation.

[6]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[7]  Plamen P. Angelov,et al.  An approach for fuzzy rule-base adaptation using on-line clustering , 2004, Int. J. Approx. Reason..

[8]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[9]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[10]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[11]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[12]  Léon Personnaz,et al.  Neural-network construction and selection in nonlinear modeling , 2003, IEEE Trans. Neural Networks.

[13]  Xiaoou Li,et al.  Online fuzzy modeling with structure and parameter learning , 2009, Expert Syst. Appl..

[14]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[15]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[16]  Uzay Kaymak,et al.  Maximum likelihood parameter estimation in probabilistic fuzzy classifiers , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[17]  Wen Yu,et al.  Restricted Boltzmann Machine for Nonlinear System Modeling , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[18]  Hak-Keung Lam,et al.  Design of stable fuzzy controller for non-linear systems subject to imperfect premise matching based on grid-point approach , 2010 .

[19]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[20]  Hisao Ishibuchi,et al.  Imbalanced TSK Fuzzy Classifier by Cross-Class Bayesian Fuzzy Clustering and Imbalance Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[21]  Martin Brown,et al.  Neurofuzzy adaptive modelling and control , 1994 .

[22]  Surajit Ray,et al.  A Nonparametric Statistical Approach to Clustering via Mode Identification , 2007, J. Mach. Learn. Res..

[23]  Norbert Stoll,et al.  Stochastic Fuzzy Modeling for Ear Imaging Based Child Identification , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[24]  Wen Yu,et al.  Randomized algorithms for nonlinear system identification with deep learning modification , 2016, Inf. Sci..

[25]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[26]  Jian-Bo Yang,et al.  Online Updating With a Probability-Based Prediction Model Using Expectation Maximization Algorithm for Reliability Forecasting , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[27]  Yoshua Bengio,et al.  Justifying and Generalizing Contrastive Divergence , 2009, Neural Computation.

[28]  Xin Zhang,et al.  Modeling of nonlinear system based on deep learning framework , 2016 .

[29]  Jun Yang,et al.  Fuzzy Model-Based Robust Networked Control for a Class of Nonlinear Systems , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[30]  Sushmita Mitra,et al.  Neuro-fuzzy rule generation: survey in soft computing framework , 2000, IEEE Trans. Neural Networks Learn. Syst..

[31]  Yoh-Han Pao,et al.  Stochastic choice of basis functions in adaptive function approximation and the functional-link net , 1995, IEEE Trans. Neural Networks.

[32]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[33]  Lotfi A. Zadeh,et al.  A Note on Z-numbers , 2011, Inf. Sci..

[34]  Zhi Liu,et al.  A probabilistic fuzzy logic system for modeling and control , 2005, IEEE Transactions on Fuzzy Systems.

[35]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[36]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[37]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[38]  Jung-Hsien Chiang,et al.  Support vector learning mechanism for fuzzy rule-based modeling: a new approach , 2004, IEEE Transactions on Fuzzy Systems.

[39]  C. L. Philip Chen,et al.  Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning , 2015, IEEE Transactions on Fuzzy Systems.

[40]  André van Schaik,et al.  Learning the pseudoinverse solution to network weights , 2012, Neural Networks.

[41]  Robert P. W. Duin,et al.  Feedforward neural networks with random weights , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[42]  Chin-Teng Lin,et al.  Neural-Network-Based Fuzzy Logic Control and Decision System , 1991, IEEE Trans. Computers.

[43]  Chia-Feng Juang,et al.  Combination of online clustering and Q-value based GA for reinforcement fuzzy system design , 2005, IEEE Transactions on Fuzzy Systems.