Data-Driven Fuzzy Modeling Using Restricted Boltzmann Machines and Probability Theory

Fuzzy modeling has many advantages over nonfuzzy methods, such as robustness with respect to uncertainties and less sensitivity to the varying dynamics of nonlinear systems. Data-driven fuzzy modeling needs to extract fuzzy rules from input and output data, and to train the fuzzy parameters of the fuzzy model. This paper takes advantages from deep learning, probability theory, fuzzy modeling, and extreme learning machines (ELMs). Restricted Boltzmann machine (RBM) and probability theory are used to overcome some common problems in data-driven modeling methods. The RBM is modified such that it can be trained with continuous values. A probability-based clustering method is proposed to partition the hidden features from the RBM. The obtained fuzzy rules have probability measurement. ELM and an optimization method are applied to train the fuzzy model. The proposed method is validated with two benchmark problems.

[1]  Xiaoou Li,et al.  Probability based fuzzy modeling , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[2]  Jian-Bo Yang,et al.  Online Updating With a Probability-Based Prediction Model Using Expectation Maximization Algorithm for Reliability Forecasting , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[3]  Norbert Stoll,et al.  Stochastic Fuzzy Modeling for Ear Imaging Based Child Identification , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[4]  Yoshua Bengio,et al.  Justifying and Generalizing Contrastive Divergence , 2009, Neural Computation.

[5]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[6]  Jung-Hsien Chiang,et al.  Support vector learning mechanism for fuzzy rule-based modeling: a new approach , 2004, IEEE Transactions on Fuzzy Systems.

[7]  Xiaoou Li,et al.  Automated Nonlinear System Modeling with Multiple Fuzzy Neural Networks and Kernel Smoothing , 2010, Int. J. Neural Syst..

[8]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[9]  André van Schaik,et al.  Learning the pseudoinverse solution to network weights , 2012, Neural Networks.

[10]  C. L. Philip Chen,et al.  Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning , 2015, IEEE Transactions on Fuzzy Systems.

[11]  Wen Yu.  A Novel Fuzzy-Neural-Network Modeling Approach to Crude-Oil Blending , 2009, IEEE Transactions on Control Systems Technology.

[12]  Paresh Chandra Deka,et al.  Fuzzy system modeling for forecasting water quality index in municipal distribution system , 2015 .

[13]  Christian Igel,et al.  Training restricted Boltzmann machines: An introduction , 2014, Pattern Recognit..

[14]  Wen Yu,et al.  Randomized algorithms for nonlinear system identification with deep learning modification , 2016, Inf. Sci..

[15]  Wen Yu,et al.  Restricted Boltzmann Machine for Nonlinear System Modeling , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[16]  Hak-Keung Lam,et al.  Design of stable fuzzy controller for non-linear systems subject to imperfect premise matching based on grid-point approach , 2010 .

[17]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[18]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[19]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[20]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  Yoh-Han Pao,et al.  Stochastic choice of basis functions in adaptive function approximation and the functional-link net , 1995, IEEE Trans. Neural Networks.

[22]  Martin Brown,et al.  Neurofuzzy adaptive modelling and control , 1994 .

[23]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[24]  Surajit Ray,et al.  A Nonparametric Statistical Approach to Clustering via Mode Identification , 2007, J. Mach. Learn. Res..

[25]  Xin Zhang,et al.  Modeling of nonlinear system based on deep learning framework , 2016 .

[26]  Jun Yang,et al.  Fuzzy Model-Based Robust Networked Control for a Class of Nonlinear Systems , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[27]  Sushmita Mitra,et al.  Neuro-fuzzy rule generation: survey in soft computing framework , 2000, IEEE Trans. Neural Networks Learn. Syst..

[28]  Korris Fu-Lai Chung,et al.  Bayesian zero-order TSK fuzzy system modeling , 2017, Appl. Soft Comput..

[29]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[30]  Kouroush Jenab,et al.  Fuzzy Reliability-Based Traction Control Model for Intelligent Transportation Systems , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Hisao Ishibuchi,et al.  Imbalanced TSK Fuzzy Classifier by Cross-Class Bayesian Fuzzy Clustering and Imbalance Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[32]  Liang Chen,et al.  Fuzzy Modeling and Adaptive Control of Uncertain Chaotic Systems , 1999, Inf. Sci..

[33]  Lotfi A. Zadeh,et al.  A Note on Z-numbers , 2011, Inf. Sci..

[34]  Jacek M. Leski,et al.  TSK-fuzzy modeling based on /spl epsiv/-insensitive learning , 2005, IEEE Transactions on Fuzzy Systems.

[35]  Robert P. W. Duin,et al.  Feedforward neural networks with random weights , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[36]  Shaocheng Tong,et al.  Adaptive Fuzzy Control for a Class of Nonlinear Discrete-Time Systems With Backlash , 2014, IEEE Transactions on Fuzzy Systems.

[37]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[38]  Zhi Liu,et al.  A probabilistic fuzzy logic system for modeling and control , 2005, IEEE Transactions on Fuzzy Systems.

[39]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[40]  Jerry M. Mendel,et al.  Interval type-2 fuzzy logic systems , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[41]  Léon Personnaz,et al.  Neural-network construction and selection in nonlinear modeling , 2003, IEEE Trans. Neural Networks.

[42]  Xiaoou Li,et al.  Online fuzzy modeling with structure and parameter learning , 2009, Expert Syst. Appl..

[43]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[44]  Shaocheng Tong,et al.  Fuzzy Adaptive Control With State Observer for a Class of Nonlinear Discrete-Time Systems With Input Constraint , 2016, IEEE Transactions on Fuzzy Systems.

[45]  Jyh-Yeong Chang,et al.  Adaptive neural-based fuzzy modeling for biological systems. , 2013, Mathematical biosciences.

[46]  Stephen A. Billings,et al.  International Journal of Control , 2004 .

[47]  Li-Xin Wang,et al.  Adaptive fuzzy systems and control , 1994 .

[48]  Johan A. K. Suykens,et al.  Wiener-Hammerstein Benchmark , 2009 .

[49]  Lutz Prechelt,et al.  Early Stopping - But When? , 2012, Neural Networks: Tricks of the Trade.

[50]  Chin-Teng Lin,et al.  Neural-Network-Based Fuzzy Logic Control and Decision System , 1991, IEEE Trans. Computers.

[51]  Plamen P. Angelov,et al.  An approach for fuzzy rule-base adaptation using on-line clustering , 2004, Int. J. Approx. Reason..

[52]  C. L. Philip Chen,et al.  A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers , 2018, IEEE Transactions on Fuzzy Systems.

[53]  Chia-Feng Juang,et al.  Combination of online clustering and Q-value based GA for reinforcement fuzzy system design , 2005, IEEE Transactions on Fuzzy Systems.

[54]  Spyros G. Tzafestas,et al.  NeuroFAST: on-line neuro-fuzzy ART-based structure and parameter learning TSK model , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[55]  Gang Chen,et al.  Deep Learning with Nonparametric Clustering , 2015, ArXiv.

[56]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[57]  Chung-Shi Tseng,et al.  Fuzzy observer-based fuzzy control design for nonlinear systems with persistent bounded disturbances , 2007, Fuzzy Sets Syst..

[58]  Junfei Qiao,et al.  An adaptive second order fuzzy neural network for nonlinear system modeling , 2016, Neurocomputing.

[59]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[60]  Xiaoou Li,et al.  Nonlinear system identification using deep learning and randomized algorithms , 2015, 2015 IEEE International Conference on Information and Automation.

[61]  James J. Buckley,et al.  Neural net solutions to fuzzy problems: The quadratic equation , 1997, Fuzzy Sets Syst..

[62]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[63]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[64]  Uzay Kaymak,et al.  Maximum likelihood parameter estimation in probabilistic fuzzy classifiers , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..